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1 Executive summary 

1.1 Introduction 

In September 2005 the National Foundation for Educational Research (NFER) in 
association with the Department for Education and Skills (DfES), the Sutton Trust and 
the College Board, began a five-year research study to examine the validity of an 
aptitude test in higher education admissions. This report describes and explores the 
relationships between scores on the SAT Reasoning Test™ (also known in brief as the 
SAT®), attainment measures (A levels and GCSEs) and background characteristics of 
the student sample. The SAT® has three measures: Critical Reading, Mathematics 
and Writing. 

1.2 Key findings 

The number of students who took the SAT® in autumn 2005 and agreed that their 
data could be used within the research was 9022. Of these SAT® participants, 8041 
students were matched to both GCSE and A level attainment data and form the basis 
of the analysis for this report (hereafter referred to as the main sample). The key 
findings to date were as follows: 

• The mean SAT® scores of the English sample were very similar to US mean 
scores and individual items functioned in a similar way in comparable samples 
of students. 

• The correlation between A level scores and the SAT® was 0.64 (i.e. high A 
level scores were generally associated with high scores on the SAT®). This 
correlation was higher than correlations of between 0.33 and 0.50 in a previous 
pilot study (McDonald et al., 2001a), most likely due to restructuring of both 
measures in the intervening years. 

• SAT® scores and total A level (or equivalent) score points were generally 
related in a similar way to a number of educational and socio-economic 
factors, including the type of institution attended, ethnicity, eligibility for free 
school meals (FSM), etc. 

• Female students had higher total GCSE and A level points scores and achieved 
significantly higher scores on the SAT® Writing component than male 
students. Male students performed significantly better on the SAT® 
Mathematics component and on the SAT® as a whole. 

• Two regression analyses, one controlling for A level total scores and one 
controlling for both A level scores and average prior attainment at GCSE, 
showed that female students, some ethnic minorities, students with special 
educational needs (SEN) and students learning English as an additional 
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language (EAL) appeared to perform less well on the SAT® than expected 
compared to default categories (i.e. boys, white students, etc). 

• Students in independent schools did better than expected on the SAT®, and 
students in FE colleges did worse, when A level results alone were taken into 
account. In both cases, they neither did better nor worse than expected when 
average prior attainment at GCSE was also taken into account. 

• Students who had gained three or more grade As at A level achieved 
significantly higher scores than the rest of the sample on all components of the 
SAT®. However there was considerable variation in the SAT® scores of this 
group offering the possibility of differentiation between high ability HE 
candidates should the SAT® prove to be a valid predictor of degree outcomes. 

• Students who achieved very high SAT® scores were not always the same 
students who achieved three or more A grades at A level and vice versa. 
Students who achieved very high SAT® scores but did not form part of the 
‘three A grades’ group tended to be male students. 

At this stage, the analysis was focussed on overall attainment and broad background 
variables. Further analyses will be carried out during the next twelve months, looking 
at differences between students according to their A level subjects and examining the 
social, educational and economic background variables using more complex statistical 
modelling of the data. This analysis will incorporate data on the attainment of the 
institutions attended, census data relating to the home postcodes of students in the 
sample and, where available, information supplied by the students themselves (e.g. 
parental education). Without higher educational outcomes it will not be possible, as 
yet, to answer the main research questions. However, this further analysis may reveal 
more about the performance of students across the SAT®, A levels and GCSEs 
according to more sensitive measures of disadvantage. 

1 .3 Structure of the report 

Section 2 describes the background to the research and the aims and objectives of the 
study. Section 3 outlines the methodologies employed and section 4 details the 
representation and background characteristics of the main SAT® sample and the 
subsequent sub-samples who took part in the student surveys in spring and autumn 
2006. 

In section 5 of this report we report the findings of the analysis of the quantitative data 
examining the relationships between the SAT® and both A level and GCSE 
attainment. These results are then explored in relation to two key research questions in 
sections 6 and 7. 

Section 8 presents the findings from the questionnaire surveys and section 9 outlines 
future potential phases of this research study and suggested milestones when further 
results will be disseminated. 
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2 Introduction 



In September 2005 the National Foundation for Educational Research (NFER) in 
association with the Department for Education and Skills (DfES), the Sutton Trust and 
the College Board, began a five-year research study to examine the validity of an 
aptitude test in higher education admissions. This report describes and explores the 
relationships between scores on the SAT Reasoning Test™ (also known in brief as the 
SAT®), attainment measures (A levels and GCSEs) and background characteristics of 
the student sample. It incorporates information given in the unpublished interim report 
(Kirkup el al., 2006) on the administration and technical functioning of the SAT®, the 
characteristics of the student sample and a preliminary analysis of the first student 
survey. 

2.1 Background 

Higher education brings considerable benefits to graduates in terms of salary, job 
security, employment opportunity, and so on. Although the number of students 
entering higher education (HE) has grown enormously in recent years, some groups 
are still under-represented. In a report into the participation in higher education over 
the period 1994-2000 (HEFCE, 2005) it was noted that young people living in the 
most advantaged 20 per cent of areas were five or six times more likely to go into 
higher education than those from the least advantaged 20 per cent of areas. The 
benefits of higher education vary according to the course studied and the institution 
attended. The demand for university places generally exceeds the supply available, 
particularly for popular courses or popular institutions. Where there are competing 
applicants, universities and other higher education institutions have to assess the merit 
and potential of each student in order to decide who to admit. 

In its report, the Admissions to Higher Education Steering Group, chaired by 
Professor Steven Schwartz (DfES, 2004), identified several issues to be addressed to 
bring about improvements to the admissions system in England and to ensure a fair 
and transparent system for all students. 1 As the system stands at present, most of the 
offers of university places to prospective students are made on the basis of predicted 
grades rather than examination results. Although the process is moving towards a 
post-qualification application system, a difficulty that will remain for admissions staff 
is that they may have to choose from an increasingly large number of highly-qualified 
candidates who achieve a string of A grades in their A level examinations. A further 
issue identified by the Schwartz report was that the information used to assess 
university applicants may not be equally reliable. Although ‘prior educational 
attainment remains the best single indicator of success at undergraduate level’ (page 



1 One recommendation of the report was the creation of a central source of expertise and advice on 
admissions issues, resulting in the Supporting Professionalism in Admissions (SPA) Programme. 
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5), it is recognised that for some students, their true potential may not be reflected in 
their examination results due to social or educational disadvantages. 

In order to differentiate further between applicants, some higher education institutions 
have already introduced or are planning to introduce additional tests or assessments 
for courses where competition for places is particularly acute. For example, applicants 
to Cambridge University may be required to attend an interview, to submit sample 
essays and, depending on the course applied for, to take the Thinking Skills 
Assessment test (TSA), the Bio-Medical Admissions Test (BMAT) or the National 
Admissions Test for Law (LNAT). Oxford University has already introduced a 
History Aptitude Test and is planning to introduce something similar for English. 
From 2007 onwards, applicants to the medical and dental schools of 24 UK 
universities will be required to sit the UK Clinical Aptitude Test (UKCAT). In 
addition Cambridge Assessment, in conjunction with the Australian Council for 
Educational Research (ACER), are piloting a test of generic reasoning and thinking 
skills (called uniTEST) with seven UK universities, with the stated aim of assisting 
higher education institutions with the student selection process. 

A recommendation of the Schwartz group was that assessment methods used within 
the admissions system should be reliable and valid. 

Admissions policies and procedures should be informed and guided by current 
research and good practice. Where possible, universities and colleges using 
quantifiable measures should use tests and approaches that have already been 
shown to predict undergraduate success. Where existing tests are unsuited to a 
course ’s entry requirements, institutions may develop alternatives, but should 
be able to demonstrate that their methods are relevant, reliable and valid. 

(p. 8) 

A levels are central to the higher education admissions process and the ability of A 
level grades to predict degree outcomes has been demonstrated using a large data set 
(Bekhradnia and Thompson, 2002), but similar evidence regarding the predictive 
validity of admissions tests or aptitude tests within the UK context appears to be 
lacking. Amongst its wider recommendations the Schwartz report encouraged the 
commissioning of research to evaluate the ability of aptitude tests to assess the 
potential for higher education. 

The principal study underpinning this current research was the pilot comparison of A 
levels with SAT® scores conducted by NFER for The Sutton Trust in 2000 
(McDonald et al., 2001a). SAT® scores were collected together with A level grades, 
prior attainment and a range of background information for 1295 students from a 
sample of high- attaining, low-attaining and selective independent schools. The study 
revealed that the SAT® was only modestly associated with A level grades, which 
indicated that the SAT® was assessing a distinct construct from A levels. (There was 
a slightly stronger association between the SAT® and GCSE attainment.) However, 
there was no evidence that the association differed according to background factors 
such as ethnicity, parental socio-economic status or overall achievement of the school. 
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The 2000 study used a shortened version of the SAT®, and further analyses explored 
the functioning of this test with English students. These showed that the SAT® 
provided a coherent assessment of verbal and math reasoning ability, and that 
individual items appeared to function similarly for English and American students. 
Item-level analyses found little evidence of bias in SAT® items between males and 
females. Whites and Asians, and overall attainment of the school. 

Although the SAT® has been relabelled as a ‘reasoning’ test rather than a test of 
‘aptitude’, it is still generally perceived as a test of academic aptitude. Implicit within 
the term ‘aptitude’ is the concept of predictive validity. The purpose of an aptitude 
test is to measure an individual’s potential for obtaining a certain goal. In this case, 
the goal is successful completion of a university course and achievement (i.e. degree 
class). If a high proportion of applicants who score well on a certain test go on to 
successfully complete their degrees, and those who score lower are somewhat less 
likely to be successful, we would say that the test has good predictive validity. 

Although the 2000 study examined the association between the SAT® and A levels, 
no recent study of the predictive power of SAT® results for university outcomes has 
been undertaken in the UK. Nor has any similar study been undertaken on the 
predictive power of the other admissions tests (subject- specific or general) that are 
being introduced by many UK organisations. The aim of the current study is to 
provide such information in due course, using the latest version of the SAT® (see 
section 3.1). For a detailed discussion of aptitude testing for university entrance see 
also the literature review conducted by McDonald et al. for the Sutton Trust (2001b). 

2.2 Objectives of the study 

The primary aim of the study is to examine whether the addition of the SAT 
Reasoning Test™ alongside A levels is better able to predict university participation 
and outcomes. Two specific issues are also to be addressed, namely: 

• Can the SAT® identify students with the potential to benefit from higher 
education whose ability is not adequately reflected in their A level results because 
of their (economically or educationally) disadvantaged circumstances? 

• Can the SAT® distinguish helpfully between the most able applicants who get 
straight As at A level? 

A secondary aim of the study is to evaluate the extent to which pastoral support and 
teaching methods affect the retention and progress of students within higher 
education. 
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3 Methodology 

In this section the main features of the SAT®, the process of contacting schools to 
participate in the research and the administration of the test itself are described. The 
methods by which students were contacted for the subsequent questionnaire surveys 
are also outlined. 

3.1 The SAT Reasoning Test™ 

The SAT Reasoning Test™ (previously known as the Scholastic Assessment Test) 
has been recently revised and now comprises three main components: Critical 
Reading, Mathematics and Writing. In the US the administration of the SAT® is split 
into ten separately timed sections, with a total test time, excluding breaks, of three 
hours and forty-five minutes. 

The Critical Reading section of the SAT® contains two types of multiple-choice 
items: sentence completion questions and passage-based reading questions. Sentence 
completion items are designed to measure students’ knowledge of the meanings of 
words and their understanding of how sentences fit together. The reading questions 
are based on passages that vary in length, style and subject and address vocabulary in 
context, literal comprehension and extended reasoning. The Mathematics section 
contains predominantly multiple-choice items but also a small number of student- 
produced response questions that offer no answer choices. Four areas of mathematics 
content are covered: number and operations; algebra and functions; geometry and 
measurement; and data analysis, statistics and probability. The new Writing section 
(first administered in the US in 2005) includes multiple-choice items addressing the 
mechanical aspects of writing (e.g. recognising errors in sentence structure and 
grammar) and a 25 minute essay on an assigned topic. 

In the English trial, no changes were made to any of the questions but one section was 
removed (a section of new items which do not contribute to the US students’ scores) 
giving a total of nine sections and an overall test time of three hours and twenty 
minutes. 

3.1.1 Contacting schools 

All maintained and independent schools and colleges in England with students in 
Year 13 were invited to participate in the research by administering the SAT® test to 
any students studying (A2) A levels. In September 2005, each institution was sent a 
letter and an information leaflet outlining the aims of the study and giving them brief 
details of what was involved for them and for their students and the feedback they 
would receive. Although their help was encouraged it was made clear that 
participation was entirely voluntary. Schools and colleges that were willing to 
participate were asked to indicate the number of students to whom they would 
anticipate administering the SAT® test in November 2005. The invitation process 
subsequently included reminding non-responding schools, twice by letter, then by fax 
and finally by telephone. At this stage the primary reason given by schools and 
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colleges that did not wish to participate was the length of the test (3 hours 20 minutes) 
- either because of difficulties fitting it into the timetable or due to concerns about 
taking students away from valuable A level study time. The other major reason cited 
was lack of time either through pressure of work or staff commitments. 

The letters inviting participation coincided with the press release to the media 
resulting in an initially positive response from schools and colleges. By the end of 
October approximately 380 schools and 50 further education colleges had agreed to 
participate with an estimated student sample of around 38,000. Reminders sent to 
non-responding schools resulted in a further number of schools/colleges agreeing to 
participate. 

In total, SAT® test materials were sent to 43,429 students at 660 institutions that had 
agreed to assist at the beginning of November. The majority of schools/colleges 
received their tests by 8 November 2005. At the end of January 2006, 294 institutions 
had returned completed SAT® tests for a total of 9207 students (see section 3.1.3). 

3.1.2 Materials 

Participating schools and colleges were sent two packs of materials: 

• a familiarisation pack containing an SAT® test preparation booklet and an 
information leaflet about the study for each student 

• a test pack containing administration guides and sufficient SAT® test booklets, 
answer forms, agreement to participate forms and HB pencils for participating 
students. 

Schools and colleges were advised that, although no formal preparation was required, 
it was recommended that students were given the familiarisation materials in advance 
of the live test administration so that they could acquaint themselves with the types of 
questions in the SAT® and how to fill in the answer sheets. Originally it had been 
planned to send the familiarisation packs to schools in October in advance of the test 
materials. However, there proved to be insufficient time to despatch them separately 
to schools and colleges. It is possible that some schools/colleges may have decided 
there was insufficient time for students to prepare for the SAT® and that it was unfair 
to administer such a novel test without thorough preparation. However, there is very 
little evidence that this was the case, and it is therefore difficult to assess whether an 
earlier, separate despatch of familiarisation materials to schools/colleges would have 
produced a higher response rate. 

The agreement to participate forms which students were asked to sign at the beginning 
of the test session collected contact details and information about the amount and type 
of preparation they had carried out. This was to enable the analysis of the results to 
take account of the level of familiarity of students with the SAT® test (see Appendix 

4). 
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3.1.3 Response rates 



Following the despatch of the materials, a sizeable number of schools and colleges 
that had originally agreed to participate contacted NFER to withdraw from the study. 
A further large number of schools returned materials unused at the end of the test 
period. The main reasons given for withdrawal or for returning materials unused were 
that time commitments prevented them from being able to administer the tests (115 
schools/colleges) or the length of the test itself (79 schools/colleges). A very small 
number of schools and colleges had concerns focussed on the content of the test, for 
example, that the mathematics component was too demanding for students not 
studying mathematics, that some sections were repetitive, or that the test had not been 
anglicised for English students. Despite attempts to contact non-responding schools 
and colleges, 170 schools/colleges failed to return test materials or to provide a reason 
for their non-re turn. 

In a large number of cases, schools and colleges participated as agreed but achieved 
much lower levels of participation than had been anticipated as illustrated by one 
school visited by a member of the research team. 

The test was done on a voluntary basis with the students given the option of 
doing the test or attending their normal classes. There had been a sixth-form 
assembly to encourage the students to take part. Of the 127 students in Year 
13, a total of 44 took the test. A number of students came to the hall on the day 
of the test, but when they realised the test was voluntary, left. 

Based on the number of students that schools and colleges had originally indicated 
might participate in the study, an average of 66 tests were sent to each participating 
institution. The average number of completed test booklets per school/college was 31. 

During the last two weeks of the autumn term, non-responding schools and colleges 
were contacted by post and subsequently by telephone to remind them to return their 
completed test materials if at all possible by the end of the term. However, in order to 
maximise the number of students in the achieved sample, some schools/colleges were 
granted additional time to carry out testing and the final date for the return of 
materials was extended to the end of the second week of January. 

At the end of January 2006, 294 institutions had returned completed SAT® tests for a 
total of 9207 students. The remaining 366 schools/colleges had returned unused 
materials or had failed to return their test materials. 

The completed answer sheets were shipped to the US and, with the exception of the 
essay section, were machine-scored by ETS. Each essay was scanned and then 
independently scored by two “readers” on-screen. If the two readers’ scores differed 
by more than one point, the essay was sent to a third chief reader for resolution. 
Scaled scores for each student and other item-level data were returned to NFER in 
March 2006. 
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3.2 Data matching 

Following the receipt of A level results from the DfES in December 2006, students in 
the SAT® sample were matched to their attainment data. As described more fully in 
section 4.1, students were excluded from the analysis if they had withdrawn from the 
study or they did not have, or could not be matched to, the two main attainment 
variables of interest, GCSE and A level data. The number of students with the 
required data was 8041 and this sample formed the basis for the analyses reported in 
section 5. 

3.3 Student survey - spring 2006 

In March 2006 the 9022 students who had taken the SAT® and had agreed to 
participate in the study were sent a 16-page questionnaire, with pre-printed names, via 
their school or college. The questionnaire asked them to provide some background 
details about their home and family circumstances and asked about their experiences 
of school or college in Years 12 and 13, their immediate plans after A levels and their 
views of higher education. 

Independent schools and further education colleges were also sent a student data form 
and asked to record the ethnic group and any special educational needs of each 
participating student. The equivalent information for students at schools in the 
maintained sector was supplied by the Department for Education and Skills (DfES). 

By the end of the spring term, schools and colleges had returned completed 
questionnaires for approximately 6250 students. A number of schools and colleges 
had failed to respond to reminders to return questionnaires or had returned complete 
sets of blank questionnaires. In order to maximise the response rate, further 
questionnaires were printed and 2121 questionnaires were despatched to the home 
addresses of the non-responding students. By the final deadline of 19 May 2006 a 
total of 6883 completed questionnaires had been received. The vast majority of the 
questionnaires were returned by schools and colleges. Of the 2121 questionnaires sent 
to home addresses, 333 were returned completed. 

From the 6883 questionnaires returned some had to be excluded from the analysis: 
duplicated questionnaires (where students had filled in copies at home and at 
school/college), a small number of spoilt questionnaires and four that were removed 
from the dataset in response to students’ requests to withdraw from the study. The 
analysis of the resultant sample of 6825 respondents was carried out in June 2006 and 
a copy of the questionnaire annotated with students’ responses is given in Appendix 1. 

3.4 Student survey - autumn 2006 

At the beginning of September 2006 a second questionnaire was sent to 8814 students 
(excluding withdrawals) who had supplied a home address for future contact. To 
thank students for their continued participation in the research, students were offered 
the opportunity to enter a prize draw to win one of five £1000 cash prizes. As a legal 
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requirement, students were not obliged to complete the survey in order to enter the 
prize draw but it was hoped that it would motivate students to do so. As well as 
completing the questionnaire students were asked to provide, or update, their email 
address so that we could contact them electronically in future. Of the 8814 
questionnaires despatched by post, approximately 150 were found to be undeliverable 
(address or addressee unknown). 

Reminders to non-respondents were sent out at the beginning of October (either by 
email or post) with the option of completing the questionnaire on-line if they preferred 
to do so. The deadline for completion of the survey was also extended until 8 January 
2007 in order to maximise the response rate. An additional text message reminder was 
sent in November to those non-responding students, who had supplied a mobile phone 
number. All students were sent a seasonal card in December, thanking them for their 
participation, giving them details of how to access more information about the 
research and reminding them of the final deadline for the survey. 

At the end of the autumn survey a total of 3352 responses had been received 
(excluding some duplicates where students had completed both paper and online 
versions) - a response rate of 38 per cent. Useable email addresses were held for 
approximately 2800 students. For the analysis, students with missing attainment data, 
who were not included in the main sample, were excluded, resulting in an autumn 
survey sample of 3177 respondents. A copy of the questionnaire annotated with 
students’ responses is given in Appendix 2. 
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4 Student samples 

4.1 Representation of the main sample 

The number of students at each phase of the project to date is shown in Figure 4.1. 



Figure 4.1 Components of the main sample 




Of the 9207 students who had completed an SAT® test, 9022 had signed a form 
agreeing for their data to be used in the study; representing 8600 students in schools 
and 422 students in FE colleges. In January 2007 the data for these students, 
excluding a small number that withdrew from the study, was matched with the 
2005/06 National Pupil Database supplied by the DfES. The dataset included A level 
data, GCSE prior attainment data and, for any student educated within the maintained 
sector. Pupil Level Annual School Census (PLASC) data. The main reasons that data 
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was missing for some students were either that they had taken the SAT® in Y12 and 
had therefore not completed their A level courses or that despite several attempts to 
match to the DfES dataset their KS4 data or their KS5 data could not be found. The 
number of students with valid data on all three main variables (SAT® scores, A levels 
and GCSEs) was 8041, hereafter referred to as the main sample. The ‘national 
population’ was derived from the same National Pupil Dataset by extracting those 
students taking two or more GCE A levels, i.e. the population that would be likely to 
be affected should a test such as the SAT® ever be introduced. 

Table 4.1 shows the main student sample by type of educational institution, as 
compared with the national population of students entered for two or more GCE A 
levels. 



Table 4.1: Main student sample by type of post-16 institution 



Main sample National population* 

Number Per cent Per cent 


Comprehensive schools to 18** 


4200 


52.2 


45.9 


Grammar schools 


1701 


21.2 


9.1 


Independent schools 


1800 


22.4 


15.0 


FE (and sixth form) colleges 


340 


4.2 


29.9 


Total 


8041 


100.0 


100.0 



* candidates entered for 2+ GCE A levels in 2005/06 (source: DfES ) 
** includes other non-selective schools 



Independent schools and grammar schools were over-represented in the sample whilst 
FE colleges were substantially under-represented. In an earlier analysis of 
participation in the SAT® trial at the institution level (i.e. comparing numbers of 
participating colleges rather than individuals) the under-representation of FE colleges 
was less pronounced. This suggests that there was a greater loss of individuals at FE 
colleges (i.e. relatively more students at FE colleges chose not to participate when 
offered the opportunity to do so). 

Comparisons of the A level and GCSE scores of the main sample against the national 
population are given in section 5.1.1. 

Background characteristics of the main sample are shown in Table 4.2. These details 
were obtained by combining information from the PLASC data for students from 
maintained schools with information supplied by individual FE colleges and 
independent schools. 
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Table 4.2: Background characteristics of the main sample 











National 






Main sample 


population* 








Valid 




Valid 






N 


per cent 


N 


per cent 


Sex 


Male 


3692 


45.9 


98625 


45.6 




Female 


4349 


54.1 


117718 


54.4 


Ethnicity 


Asian or Asian British 


670 


9.1 


7799 


6.9 




Black or Black British 


117 


1.6 


2243 


2.0 




Chinese 


116 


1.6 


996 


0.9 




Mixed 


145 


2.0 


1392 


1.2 




White 


6212 


84.4 


93732 


83.2 




Other 


104 


1.4 


6499 


5.8 


SEN 


No provision 


7437 


97.3 


114818 


97.9 




School Action (A) 


137 


1.8 


1632 


1.4 




School Action Plus (P) 


35 


0.5 


474 


0.4 




Statement of SEN (S) 


32 


0.4 


384 


0.3 


FSM eligibility 


No 


5953 


96.1 


114058 


97.2 


Yes 


243 


3.9 


3250 


2.8 


Total 




8041 


100 


216343 


100 



* candidates entered for 2+ GCE A levels in 2005/06 (source: DfES) 
Valid percentages exclude missing data. 

Due to rounding, percentages may not sum to 100. 



As can be seen from Table 4.2, there were more female students in the main sample 
than male students (54 per cent and 46 per cent respectively), in line with the national 
population of A level entrants. 

For a small number of students in the main sample, and for a considerable numbers of 
students in the national sample, information on ethnicity, special education needs and 
eligibility for free school meals was missing. In the national figures approximately 
one third of the missing data was for students from the independent sector. Comparing 
those for whom information was available, there were slightly more Asian and 
Chinese students in the main sample compared to the national population of A level 
students and slightly fewer Black students. The figures for students with special 
educational needs and those eligible for free school meals may be somewhat distorted 
due to the large numbers of students in the national sample for whom data was 
missing. Approximately three per cent of the main sample were known to be eligible 
for free school meals and 2.5 per cent were known to be on the register of special 
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educational needs. The figures for these categories are slightly higher in the table 
where missing data has been excluded in order to enable comparisons with the 
national data. 

Attainment data and SAT® data for the main sample is reported in Section 5. For 
those students in the main sample now in higher education, degree outcomes will be 
supplied by HEFCE (Higher Education Funding Council for England) at the end of 
their three year courses irrespective of contact with them in the interim period. 
Therefore degree outcome data for the majority of the main sample will be available 
to examine the issue of predictive validity. Collecting qualitative data is more 
complex as outlined in section 4.2 below. 

4.2 Student survey sub-samples 

At various stages during the research additional information from participating 
students may be collected via questionnaire surveys. To date, two such surveys have 
been carried out, the responses to which are shown in Figure 4.2. 

Approximately three-quarters of the original SAT® sample (6825 students) responded 
to the first survey in spring 2006, which was administered through schools and 
colleges. However, the second survey in autumn 2006 required students to be 
contacted individually. Recognising the difficulties of maintaining contact with a 
highly mobile population, students were contacted in a number of different ways 
during this second survey and a prize draw incentive was offered (see section 3.4) A 
response rate of 38 per cent (of potential respondents) was achieved for the autumn 
survey, although some of these were subsequently excluded because of missing 
attainment data. 
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Figure 4.2: Components of the sub-samples 




A comparison of the background characteristics of the main sample and the two 
questionnaire survey samples is presented in Table 4.3. In this table, students who 
took part in the spring survey but were not subsequently included in the main sample 
were excluded. 

Of the main sample of 8041 students, 77 per cent responded to the spring survey, 40 
per cent to the autumn survey and 34 per cent (2750 students) to both surveys. For 
future potential surveys the current database contains email addresses for 
approximately 2800 students and postal contact details for a further 5800. 
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Table 4.3: Background characteristics of the survey sub-samples 







Main 

per cent 
(n = 8041) 


Survey samples 

Spring Autumn 

per cent per cent 

(n = 6189) (n = 3177) 


Sex 


Male 


46 


44 


37 




Female 


54 


56 


63 


Ethnicity 


Asian or Asian British 


8 


8 


6 




Black or Black British 


1 


1 


1 




Chinese 


1 


2 


1 




Mixed 


2 


2 


2 




White 


77 


78 


81 




Other 


1 


1 


1 




Missing 


8 


7 


8 




No 


70 


68 


70 


EAL 


Yes 


7 


7 


6 




Missing 


23 


25 


24 


SEN 


No provision 


92 


93 


93 




School Action (A) 


2 


2 


1 




School Action Plus (P) 


0 


1 


0 




Statement of SEN (S) 


0 


0 


1 




Missing 


5 


4 


5 


FSM 


No 


97 


97 


97 


eligibility 


Yes 


3 


3 


3 




Comprehensive schools 


52 


49 


49 


Post- 16 
institution 


Grammar schools 


21 


22 


23 


type 


Independent schools 


22 


25 


23 




FE / sixth form colleges 


4 


5 


6 


Total 




100 


100 


100 



Due to rounding, percentages may not sum to 100. 

School Action: Interventions provided by the teacher and / or special needs co-ordinator that are 
additional to or different from those provided as part of the school’s usual differentiated curriculum. 
School Action Plus: the next stage in the SEN support process when external services are involved. 



16 








Although smaller in size, the survey samples appear to be reasonably similar to the 
main sample in terms of their background characteristics. The percentage of male 
respondents was lower in the autumn survey than in the spring survey. The autumn 
questionnaires were sent to individual home addresses whereas the spring survey was 
administered via schools and colleges. Also an incentive was offered to students in the 
autumn survey which may have impacted differently on male and female students. 

There were also fewer Asian or Asian British respondents in the autumn survey 
compared to the percentage of Asian students in the main sample and the spring 
survey. The percentage of respondents from comprehensive schools in both survey 
samples was slightly less than the percentage in the sample as a whole whereas the 
percentage of respondents from FE colleges was slightly higher in the surveys. 
Although similar to the full sample in terms of their background characteristics, a 
comparison of respondents and non-respondents to the spring and autumn survey 
showed that the mean total A level points of respondents (872 in the spring survey and 
930 in the autumn) were significantly higher than the mean points score of non- 
respondents (770 and 795 respectively). This is perhaps to be expected given that the 
subject of the research is more likely to be salient for high achieving students who are 
either in, or intending to enter, higher education. 

For a brief discussion of the possible impact of the prize draw incentive relating to the 
autumn survey see section 4.2.2. 

4.2.1 Additional background characteristics from the spring 
survey 

The spring 2006 questionnaire provided an opportunity to ask students for additional 
background information on their home circumstances. In subsequent phases of the 
research, it is hoped to relate the information collected from these surveys to the 
university destinations and degree outcomes of students in the main sample. As this 
data was analysed before the A level data was received, findings in this section, and in 
section 8, are based on the 6825 students who responded, of whom 6189 students are 
included in the main sample. 

Of the 6825 students, 86 per cent had always lived in the UK. Of those who had not, 
the average length of stay in the UK was nine years. Approximately 15 per cent of the 
students indicated that they spoke a language other than English at home'. There were 
a total of 53 different languages spoken. The most commonly spoken home languages 
were Mandarin and Cantonese, Gujerati, Punjabi and Urdu. 



2 This figure is somewhat higher than the percentage of students known to be learning English as an 
additional language (see Table 4.3). This may be due to missing data or to some students listing another 
language spoken at home rather than the language that is usually spoken at home. 
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The majority of students (86 per cent) lived in a property owned by a family member 
and six per cent lived in rented accommodation. 

Table 4.4 provides information on the household structures in which students lived. 



Table 4.4: ‘Who do you live with most of the time?’ 





Number 


Per cent 


Father only 


154 


2.3 


Mother only 


1006 


14.7 


Mother and father (no other adults) 


4853 


71.1 


1 natural and 1 step parent/partner 


389 


5.7 


Other 


320 


4.7 


Missing 


103 


1.5 


Total 


6825 


100.0 



Due to rounding , percentages may not sum to 100. 



The majority of students came from two parent families and a significant number 
were looked after by their mother only. Included in the Other category were students 
who lived with one or more parent and also another adult and those students who 
lived with an adult other than a parent, such as a grandparent. Also included in the 
Other category were approximately 50 students attending boarding schools, 12 
students who lived with foster parents and two students in care. 

Approximately 35 per cent of the students’ mothers and 41 per cent of the students’ 
fathers had studied at a higher education institution. Many of the students had a 
sibling who had attended or was studying at a university (36.7 per cent). However, 
one third of the sample had no close relative who had gone to university. 

The students were asked to write the occupation of the parent who earned the most, 
describe briefly what this entailed and say whether this parent had responsibility for 
supervising other members of staff. The parental occupations were then coded using 
Standard Occupational Classification 2000 (SOC 2000). The codes were then grouped 
as shown in Table 4.5. 
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Table 4.5: Parental occupation by socio-economic classification 





Number 


Valid 


National 




per cent 


per cent* 


1.1 Large employers and higher 
managerial occupations 


767 


12.8 


► 26.2 


1.2 Higher professional occupations 


1332 


22.3 




2 Lower managerial and professional 
occupations 


1718 


28.7 


32.6 


3 Intermediate occupations 


576 


9.6 


14.3 


4 Small employers and own account 
workers 


616 


10.3 


7.0 


5 Lower supervisory and technical 
occupations 


342 


5.7 


5.0 


6 Semi-routine occupations 


376 


6.3 


10.3 


7 Routine occupations 


223 


3.7 


4.6 


8 Unemployed (housewives, students, 
retired, etc) 


31 


0.5 


0.0 




5981 


100.0 


100.0 


Missing / uncodeable occupations 


844 






Total 


6825 







Due to rounding, percentages may not sum to 100. 

* entrants to UK HEIs 2004/05 (DfES analysis ofHESA student record 2004/05) 



Over half of the spring survey sample had at least one parent working in professional 
or managerial occupations. Coverage by parental occupation is reasonably well 
matched to the national profile of HE entrants, although the HE population does not 
reflect that of society as a whole. 

The spring 2006 questionnaire also asked the students to estimate how many books 
were in their home. In previous studies (e.g. Kirkup el al., 2004), this has been found 
to be a useful indicator of socio-economic status. 
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Table 4.6: The number of books in students’ homes 



No. of books 


Number Per cent 


None (0 books) 


47 


1 


Very few (1-10 books) 


224 


3 


Enough to fill one shelf (11-50 books) 


780 


11 


Enough to fill one bookcase (51-100 books) 


1079 


16 


Enough to fill two bookcases (101-200 books) 


1423 


21 


Enough to fill three or more bookcases (more than 
200 books) 


3193 


47 


Missing 


79 


1 


Total 


6825 


100 


Due to rounding, percentages may not sum to 100. 






The relationships between background variables, SAT® scores and attainment are 


explored in section 5 and responses to the spring survey are presented 


in full in 


Appendix 1. 






4.2.2 Additional information from the autumn survey 




The autumn 2006 survey provided information on the post A level destinations for 
approximately 40 per cent of students from the main sample. The indicated 
destinations or plans of these 3177 respondents for the 12 months commencing 
autumn 2006 are presented in Table 4.7. 


Table 4.7: Post-school destinations of the 
sample 


autumn 2006 


survey 


Number Per cent 


Starting at university / other HEI in autumn 2006 


2455 


77 


Deferred / guaranteed place at university / other HEI 
in autumn 2007 (gap year students) 


285 


9 


Apprenticeship / Employment 


190 


6 


Education (below HE) 


108 


3 


Unemployed / Break from work and study 


84 


3 


Missing 


55 


2 


Total 


3177 


100 



Due to rounding, percentages may not sum to 100. 
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Although these figures represent the destinations of only around 40 per cent of the 
main sample, the proportion of this sub-sample starting a course at an HEI in 2006 (77 
per cent) corresponds closely to the equivalent figure from the spring survey, when 
around 75 per cent of respondents indicated that this was what they intended to do. A 
more detailed examination of the HE destinations of students in the sample will be 
presented in the 2008 report. 

A further question asked those respondents commencing an HE course in 2006 to 
indicate the duration of their course, as shown in Table 4.8. 



Table 4.8: Course duration for 2006 HE entrants in autumn 2006 
survey sample 





Number 


Per cent 


1 year 


48 


2 


2 years 


22 


1 


3 years 


1494 


61 


4 years 


704 


29 


more than 4 years 


180 


7 


Missing 


7 


0 


Total 


2455 


100 



Due to rounding, percentages may not sum to 100. 



If similar proportions to those in Table 4.7 are true of the main sample as a whole, 
these figures suggest that in a year’s time it will be possible to explore the 
relationships between SAT® scores, attainment and university destinations for 
approximately 86 per cent of the sample, around 6900 students. Taking into account 
the figures in Table 4.8, it is likely that, due to the number of students taking a gap 
year and those taking courses of more than 3 years duration, the number of students in 
the main sample completing their degrees in the academic year 2008/09 is likely to be 
around 3800, with a further 2200 students completing a degree one year later. 

As reported in section 3.4 potential autumn survey participants were offered the 
opportunity to enter a prize draw in the hope that this would motivate them to 
complete a questionnaire (although they were not obliged to do so). It had been 
estimated that the response rate without an incentive would be approximately 33 per 
cent and that the incentive might increase this to up to 50 per cent of potential 
respondents. In the event the achieved response rate, as reported in the same section, 
was 38 per cent. It is difficult to assess the impact of the incentive, although with 
hindsight perhaps the initial estimates had been over-optimistic given the age and 
mobility of the sample. 

However, on one measure at least it does not appear that the impact of the incentive 
was related to economic need as there was no significant difference in the autumn 
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survey between respondents and non-respondents grouped by their eligibility for free 
school meals - see Table 4.3 on page 17. If the incentive had been more effective with 
less advantaged participants, one would have anticipated that the proportion of 
respondents to the autumn survey eligible for free school meals would have increased 
relative to the proportion in the sample as a whole. 

Survey respondents were also asked to indicate which of four potential incentives 
would most encourage them to complete a survey in the future. For these respondents 
a music or book token (value £5) sent with the questionnaire was the incentive 
indicated as being most likely to encourage them to participate. However, it is 
difficult to know whether this would also be the case with non-respondents. It would 
also be a much more expensive option than the prize draw. 

Responses to the autumn survey are presented in full in Appendix 2. 
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5 Relationships between SAT® scores 
and attainment 



In this section the analyses of the attainment and aptitude data are described and the 
relationships between the various measures are explored. The main study variables for 
each participant were their total A level score, their total GCSE score and their SAT® 
scores for Critical Reading, Mathematics and Writing. A description of each of these 
variables is given at the beginning of the relevant section below. 

5.1 Descriptive statistics 

5.1.1 Attainment data 

In the analyses that follow, the attainment data for students in the sample was taken 
from a dataset supplied to the NFER by the DfES. The A level score used in the 
analyses was the total QCA point score for all Level 3 qualifications approved as A 
level equivalences. The points awarded under this system differ from the UCAS point 
scores: for example a GCE A level grade A is equivalent to 270 points (compared to 
120 points using the UCAS tariff). For prior attainment the GSCE variables used in 
the analyses were the total KS4 point score and the average KS4 point score. Again 
the GCSE point scores are based on a new system (developed by QCA) in which a 
GCSE grade G is equivalent to 16 points and an A* grade is equal to 58 points. 
Further details of the scoring systems for both KS4 and KS5 qualifications and 
information about the discounting process (used to avoid double counting of 
qualifications such as GCE A and AS levels) can be found on the DfES website 
(DfES, 2006). 

Table 5.1 shows the sample and national means for the key attainment measures and 
Figures 5.1 and 5.2 compare the score distributions of A level total scores and GCSE 
average scores for the main sample with those of the national population. 

The main sample spans a wide range of ability but with a score distribution slightly 
skewed towards the upper range compared to the national population of A level 
entrants taking 2 or more GCE A levels. The mean A level points score of the main 
sample was 849. The mean A level total score for the national sample was slightly 
lower at 808. 

Similarly the prior attainment of the main sample was slightly higher (an average 47 
GCSE points) than that of the national population (46 points). The differences in 
means of the sample and the population are statistically significant. However, 
although the distribution of the main sample is skewed towards the high end (probably 
because of the number of students from grammar and independent schools) it broadly 
covers the same range as the population. The sample therefore contains sufficient 
cases from all areas of the population to enable reasonable conclusions to be drawn. 
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Table 5.1: Mean attainment scores - main sample 





Main sample 
Mean s.d. 


National population* 
Mean s.d. 


Total A level (or L3 
equivalent) point score 


848.6 


260.4 


808.4 


235.8 


Total GCSE point score 


489.9 


80.1 


469.0 


107.6 


Average GCSE point score 


47.4 


6.0 


46.4 


5.5 




n = 


8041 


max n 


= 216343 



Values significantly different at the 5 per cent level are shown bold and in italics. 

* 2005/06 GCE A level entrants taking 2+ A levels from the dataset supplied by DfES 



Figure 5.1: Banded A level score distributions for main sample and 
national population 



Total KS5 point score 




0-200 201 -300 301 -400 401 -500 501 -600 601 -700 701 -800 801 -900 901 - 1001 - 1101 - 1201 + 

1000 1100 1200 
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Figure 5.2: Banded GCSE average score distributions for main 
sample and national population 



Average KS4 point score 




0 - 32.5 35 - 37.49 37.5 - 40 - 42.49 42.5 - 45 - 47.49 47.5 - 50 - 52.49 52.5 - 55 - 57.49 57.5 + 

39.99 44.99 49.99 54.99 



5.1.2 SAT® data 

The raw SAT® scores of students who participated in the English trial were converted 
to scaled scores using the scoring metric of the US SAT®. SAT® scores for the main 
three components (Critical Reading, Mathematics and Writing) are each reported on a 
scale from 200 to 800. The multiple choice writing section counts for approximately 
70 per cent and the essay counts for approximately 30 per cent of the total writing raw 
score, which is used to calculate the 200 to 800 score. Additional sub-scores are 
reported for the essay (ranging from 2 to 12) and for multiple-choice writing questions 
(on a 20 to 80 scale). Each essay is independently scored 1 to 6 by two “readers”. 
These readers’ scores are combined to produce the 2 to 12 scale. (If the two readers’ 
scores differ by more than one point, the essay is sent to a chief reader for resolution. 
Essays that are not written on the essay assignment, or which are considered illegible 
after several attempts at reading, receive a score of 0.) The US mean or average scaled 
score for Critical Reading, Maths, and Writing is usually about 500. 

Table 5.2 shows the means obtained on each of the main components of the SAT® 
and Figures 5.3 to 5.5 show the total score distributions for the Critical Reading, 
Mathematics and Writing components. For comparison purposes, the means and score 
distributions for over 1.4 million students in the US 2006 College-bound Seniors 
cohort are given (College Board, 2006). 
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Table 5.2: Mean SAT® scores - main sample and US 2006 cohort 



SAT® component 


Mean 


s.d. 


US mean 


US s.d. 


Critical reading 


500 


115 


503 


113 


Mathematics 


500 


116 


518 


115 


Writing 


505 


88 


497 


109 


Writing: multiple-choice 


49.1 


9.4 


n/a 


n/a 


Writing: essay 


7.7 


1.5 


n/a 


n/a 




n = 


8041 


n= 1,465,744 



As can be seen in the above table the means achieved are roughly comparable with 
US means, averaged over a number of administrations throughout the year using 
different versions of the SAT®. The means of a US sample that took the identical 
version to that of the English sample in March 2005 were considerably higher (526, 
544 and 525 respectively). However, that was the first administration of the SAT® to 
include the essay component and attracted a very able group of candidates. (See 
Appendix 4 for a more detailed comparison of the two groups.) 

Figure 5.3: Total score distribution for critical reading 



SAT® Reading Score distributions for England and US 




Score range 
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Figure 5.4: Total score distribution for mathematics 



SAT® Mathematics Score distributions for England and US 




Figure 5.5: Total score distribution for writing 

SAT® Writing Score distributions for England and US 




Score range 



Much of the analysis of the SAT® data was carried out in summer 2006 following the 
receipt of the scores from the US. Descriptive statistics for the full UK SAT® sample 
of 9022 students (including means and score distributions for sub-groups) are 
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presented in Appendix 3. A full description of the analyses examining the functioning 
of the SAT®, as a complete test and at the individual item level, is presented in 
Appendix 4. Where data was available these analyses included comparisons with a 
sample of approximately 290,000 US students who took exactly the same version of 
the SAT® to see if there were any differences in performance that might indicate that 
the SAT® test was unsuitable for the English sample. Overall these results indicated 
that the individual SAT® items functioned in a similar way for the English and US 
samples. 

Tables 5.3 to 5.7 give breakdowns of the main study variables - SAT® scores, A 
level scores and GCSE scores - by background variables. In most cases the 
differences between groups are statistically significant; which is to be expected given 
the size of the sample. 



Table 5.3: Main study variables by gender 





Male 


Female 


Total 


Number of cases 


3692 


4349 


8041 


% of cases 


46% 


54% 


100% 


Mean A level total score 


825.2 


868.5 


848.6 


Mean GCSE total score 


485.9 


493.3 


489.9 


Mean SAT® score 


505.3 


498.4 


501.55 


SAT® reading 


497.6 


501.7 


499.8 


SAT® mathematics 


523.3 


480.3 


500.0 


SAT® writing 


494.9 


513.3 


504.8 


Writing: multiple-choice 


48.5 


49.6 


49.1 


Writing: essay 


7.4 


7.9 


7.7 



Values significantly different at the 5 per cent level are shown bold and in italics. 



It is interesting to note that females out-perform males, on average, at GCSE and A 
level and in the SAT® writing test, but the reverse is true for SAT® mathematics and 
in the SAT® test as a whole. It may be that this is due to the different mode of 
assessment, or else to the different subject balance. 

The differences between male and female students on the various SAT® components 
are similar to the most recent results for students in the USA, where male students 
generally outperform female students in mathematics but do less well in writing 
(College Board, 2006). 
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Table 5.4: Main study variables by free school meal eligibility 





Not eligible 


Eligible 


Eligibility unknown 


Number of cases 


5953 


243 


1845 


% of cases 


74% 


3% 


23% 


Mean A level total score 


828.0 


665.5 


939.3 


Mean GCSE total score 


490.9 


454.8 


491.2 


Mean SAT® score 


495.0 


439.4 


530.8 


SAT® reading 


492.7 


444.3 


530.0 


SAT® mathematics 


494.4 


424.0 


528.4 


SAT® writing 


498.0 


450.0 


534.1 


Writing: multiple-choice 


48.4 


43.6 


52.0 


Writing: essay 


7.6 


7.1 


8.0 



Values significantly different at the 5 per cent level are shown bold and in italics. 



Clearly students known to be eligible for free school meals under-perform on all 
outcomes, on average compared with the other categories. Most of the students in the 
‘eligibility unknown’ category were in independent schools. Therefore this 
classification will be confounded with the institution type classification presented in 
Table 5.7. In other words, the salient characteristic of such students in respect of 
attainment is probably that they were in independent schools and not that their 
eligibility for free school meals was unknown. 
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Table 5.5: Main study variables by ethnicity 





Asian 


Black 


Chinese 


Mixed 


White 


Other 


Not 

Known 


Number of cases 


670 


117 


116 


145 


6212 


104 


677 


% of cases 


8.3% 


1.5% 


1.4% 


1.8% 


77.3% 


1.3% 


8.4% 


Mean A level total 
score 


766.7 


690.1 


956.0 


836.9 


851.5 


793.9 


922.9 


Mean GCSE total 
score 


477.3 


456.9 


477.6 


487.2 


491.8 


505.1 


491.1 


Mean SAT® score 


454.9 


443.8 


484.8 


493.8 


506.4 


485.3 


520.4 


SAT® reading 


436.6 


436.8 


445.1 


493.2 


507.5 


478.1 


516.6 


SAT® mathematics 


474.5 


438.3 


566.6 


489.4 


500.6 


499.6 


521.5 


SAT® writing 


453.6 


456.2 


442.8 


498.9 


511.0 


478.3 


523.1 


Writing: multiple- 
choice 


43.4 


44.3 


43.0 


47.9 


49.8 


46.2 


50.8 


Writing: essay 


7.4 


7.2 


6.9 


7.9 


7.7 


7.5 


7.9 



Rows which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at the 5 per cent level. 



Clearly there are different patterns of performance for different ethnic groups. The 
most striking example is the Chinese group, who, on average, are the highest group in 
terms of A level and SAT® mathematics scores, but perform more poorly in SAT® 
reading and are the lowest group in SAT® writing. Asian and Black students achieved 
very similar mean SAT® scores whereas there was a much larger difference in A 
level scores between the two groups. 
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Table 5.6: Main study variables by English as an additional 
language (EAL) status 





Not EAL 


EAL 


EAL status 
unknown 


Number of cases 


5624 


572 


1845 


% of cases 


70% 


7% 


23% 


Mean A level total score 


830.1 


737.9 


939.3 


Mean GCSE total score 


490.9 


476.3 


491.2 


Mean SAT® score 


497.8 


443.9 


530.8 


SAT® reading 


497.7 


422.9 


530.0 


SAT® mathematics 


494.0 


467.8 


528.4 


SAT® writing 


501.7 


441.0 


534.1 


Writing: multiple-choice 


48.8 


42.3 


52.0 


Writing: essay 


7.6 


7.2 


8.0 



Values significantly different at the 5 per cent level are shown bold and in italics. 



A similar comment to that made in respect of free school meal eligibility (Table 5.4) 
can be made here. Those whose EAL status is unknown are the same individuals, 
mostly from independent schools, and therefore this classification will be confounded 
with the institution type classification shown in Table 5.7. As might be expected, 
differences between the performance of EAL and non-EAL students are least for the 
SAT® mathematics component. 
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Table 5.7: Main study variables by institution type 



Comprehensive 

school 


Grammar 

school 


Independent 

school 


FE college 


Number of cases 


4200 


1701 


1800 


340 


% of cases 


52% 


21% 


22% 


4% 


Mean A level total score 


755.0 


977.5 


942.1 


864.8 


Mean GCSE total score 


478.8 


515.1 


494.0 


479.0 


Mean SAT® score 


472.6 


543.0 


533.7 


481.4 


SAT® reading 


470.3 


541.3 


533.2 


480.0 


SAT® mathematics 


467.9 


550.3 


531.6 


478.4 


SAT® writing 


479.6 


537.4 


536.4 


485.7 


Writing: multiple-choice 


46.5 


52.6 


52.2 


47.1 


Writing: essay 


7.4 


7.9 


8.0 


7.5 



Rows which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at the 5 per cent level. 



The general order of mean outcome scores shown above has comprehensive schools 
with the lowest scores, with FE colleges just above, followed by independent schools 
and grammar schools. In most cases there is no significant difference between 
comprehensive schools and FE colleges, or between independent and grammar 
schools. The exception is the total A level score, where the FE college mean is 
significantly higher than that for comprehensive schools, despite both having similar 
GCSE total scores. 

Finally in this section, Table 5.8 shows the main study variables for a sub-sample of 
students who responded to the autumn survey and for whom we have information 
concerning their current activities. This information will be updated in the 2008 report 
when more complete HE destination data will have become available. 
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Table 5.8: Main study variables by current activity 





HE 


Gap year 


Other 


Number of cases 


2434 


306 


427 


% of cases 


77% 


10% 


13% 


Mean A level total score 


955 


881 


824 


Mean GCSE total score 


516 


500 


488 


Mean SAT® score 


538 


523 


509 


SAT® reading 


540 


530 


514 


SAT® mathematics 


537 


511 


498 


SAT® writing 


535 


527 


515 



Rows which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at the 5 per cent level. 



Those students already in higher education achieved significantly higher A level, 
GCSE and average SAT® scores compared to gap year students - they in turn 
performed significantly better than other respondents not currently considering HE, 
i.e. those in employment, education below HE, unemployed, etc. In terms of the 
separate SAT® components, there was no significant difference between the reading 
and writing scores of HE and gap year students. There was also no significant 
difference between the SAT® mathematics scores of gap year and other respondents. 
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5.3 Exploring the relationships between the main 
study variables 

Table 5.9 below displays the correlations between the GCSE and A level scores and 
between GCSE and A level scores and each of the SAT® scores. 



Table 5.9: Correlations between 
SAT® 


GCSE and A 


level scores and 




A level total 
score 


GCSE total 
score 


Average GCSE 
score 


Mean SAT® score 


0.64 


0.54 


0.70 


SAT® reading 


0.55 


0.46 


0.59 


SAT® mathematics 


0.54 


0.48 


0.60 


SAT® writing 


0.57 


0.48 


0.64 


Writing: multiple-choice 


0.55 


0.47 


0.62 


Writing: essay 


0.32 


0.25 


0.34 


A level total score 




0.58 


0.76 


GCSE total score 






0.70 



Correlations significantly different from zero at the 5 per cent level are shown bold and in italics. 



In the above table it is clear that the correlation between total SAT® score and A level 
total score is somewhat higher than with GCSE total score, but that the highest 
correlation with total SAT® is average GCSE score. Correlations with the different 
components of the SAT® are similar, except for the essay element which has much 
lower correlations with GCSE and A level outcomes (probably at least partly because 
of the relatively restricted range of the essay score). 

The correlation of total A level points with average GSCE score is higher than with 
the total GCSE score. It is likely that this is because the number of GCSEs entered can 



3 Correlation: a measure of association between two measurements, e.g. between size of school and the 
mean number of GCSE passes obtained by each pupil. A positive correlation would occur if the 
number of passes increased with the size of the school. If the number of passes decreased with size of 
school there would be a negative correlation. Correlations range from -1 to +1 (perfect negative to 
perfect positive correlations); values close to zero indicate no linear association between the two 
measures. 
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vary widely and does not always reflect the ability of the student whereas at A level 
there is far less variation in the number of A levels attempted. 

The higher correlation between SAT® and average GCSE score than between SAT® 
and A levels is in accordance with previous findings. In the pilot SAT® study carried 
out in 2000 in three groups of low-achieving, high-achieving and independent 
schools, (McDonald et al., 2001a), the correlation between the SAT® and mean 
GCSE grade was higher in all types of schools than between the SAT® and mean A 
level grade. At that time the correlations between mean GCSE grade and total SAT® 
score for high-achieving and low-achieving schools were 0.62 and 0.58 respectively 
and the correlations between SAT® score and mean A level grade were 0.45 and 0.50. 

The SAT® as a whole has undergone some change since 2000, particularly the 
introduction of the writing components, and therefore one would expect a higher 
correlation between total SAT® scores and A levels than previously. Also there have 
been considerable changes to the A level system since the pilot; a greater number of 
subjects are now studied at A level and the structure of such courses is modular. 

The high correlations between SAT® scores and attainment at GCSE and A levels are 
not unexpected given that each of these is measuring overall educational ability, albeit 
measuring different aspects and in different ways. Research generally shows similarly 
high correlations between different measures of educational ability. For example, 
Thomas & Mortimore (1996) found correlations of 0.72, 0.67 and 0.74 between 
Cognitive Abilities Test (CAT) scores in Year 7 and GCSE total points score, GCSE 
English grades and GCSE mathematics grades respectively. Correlations between 
measures of educational ability are also generally higher when such measures are 
administered in close proximity to one another, as is the case with the SAT® and the 
A level examinations. 

The relationship between A levels and SAT® scores is complicated in that each of 
these measures is associated with prior attainment at GCSE. Controlling for average 
attainment at GCSE, the partial correlation between SAT® and A levels was 0.23. 
This suggests that, although SAT® and A levels are highly correlated, the underlying 
constructs that are being measured are somewhat different. This may indicate a 
potential for the SAT® to add to the prediction of HE outcomes from A levels, 
although the increment is likely to be relatively small. Whether this is indeed the case 
will not be known until such outcomes are available for students in the sample. 

As yet, there has been insufficient time to carry out an analysis looking at the 
correlations between SAT® scores and the attainment of students grouped by the 
subjects studied at A level. This analysis will examine in more depth the extent to 
which A levels, in particular subjects, and the various components of the SAT® are 
measuring similar constructs. 

Finally, an initial analysis was carried out to examine the relationships between the 
main study variables, a wide variety of background characteristics and post A level 
activities. From the autumn survey responses, participants were categorised as 2006 
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HE entrants, gap year students and Not-HE. Logistic regression models 4 were run to 
predict membership of these different groups based on attainment and background 
characteristics. When 2006 HE entrants were compared with all others, students from 
independent schools were less likely to be in the 2006 HE group compared to students 
from comprehensives. However the independent school variable was not significant 
when 2006 HE entrants and gap year students were together compared to the Not-HE 
group. A possible explanation is that students from independent schools are likely to 
go into HE but have the financial resources or support to enable them to take a gap 
year and therefore they are less likely to go to HE immediately. A small number of 
other variables were significant in the models but these were less easy to interpret. 
More sophisticated analyses may be required once we have information about the 
destinations of a larger number of students in the main sample. 

The following two sections present data analysis which aims to explore two different 
issues: 

• To what extent, and for whom, does the SAT® add relevant information about 
performance? 

• Does the SAT® help to distinguish between those with three grade As or 
above at A level? 



4 Logistic regression is a technique which aims to predict the probability of an individual belonging to a 
certain group based on a range of background characteristics. 
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6 Disadvantaged students: 

Can the SAT® identify students with the potential to 
benefit from higher education whose ability is not 
adequately reflected in their A level results because of 
their (economically or educationally) disadvantaged 
circumstances? 

While it is not yet possible to address this question directly without having any 
measure of higher education outcomes, it is possible to look at a couple of related 
questions: 

• What are the characteristics of students with more inconsistent performance 
measures? 

• Do particular types of student perform better in the SAT® than would be 
predicted from their A level (and possibly GCSE) results? 

In the past, an argument that has been put forward for the SAT® is that it measures 
aptitude rather than attainment and is therefore less dependent on a student’s socio- 
economic background and schooling - although the evidence does not unequivocally 
support this view (see McDonald et al., 2001b). However, if this is the case, and 
assuming A levels do not adequately reflect the ability of some disadvantaged 
students, it would be likely that such students might exhibit inconsistent performance, 
i.e. perform better on the SAT® than in their A levels. 

The thinking here is that it is only those with inconsistent performance who would be 
likely to benefit from the introduction of SAT®; those whose overall performance is 
well-reflected by A level would not be affected by the additional measure. 

To explore this question a measure of “inconsistency” was devised across the three 
available performance measures, namely: 

• mean SAT® score 

• A level total point score 

• GCSE total point score. 

Each of the above is measured on a different scale, so in order to compare scores 
across the different outcomes these were ‘standardised’ - each was converted to a 
scale with a mean value of 100 and standard deviation of 15 across the 8041 cases in 
the main sample. Extreme outliers were excluded by truncating the standardised 
scores at 59 and 141. Once this is done, a highly ‘consistent’ student would be 
expected to score approximately the same standardised score across all the measures - 
if they scored 85 on their GCSE standardised score (one standard deviation below the 
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mean) they would score about the same on all the others. An ‘inconsistent’ student 
would have more variability in their standardised scores. 



By taking an average score to represent performance on the SAT® as a whole, any 
inconsistencies across the different components of the SAT® reported previously, for 
example by Chinese students (see Table 5.5), were reduced. 

The simplest measure of inconsistency was taken as the range of standardised score 
values, i.e. the difference between the highest and lowest standardised score for each 
student. Table 6.1 gives an example of this calculation for two hypothetical students. 



Table 6.1: Calculation of inconsistency measure for two students 



Standardised scores 


Student A 


Student B 


mean SAT® score 


83 


72 


A level total point score 


88 


85 


GCSE total point score 


84 


119 


Highest standardised score 


90 


119 


Lowest standardised score 


83 


72 


Inconsistency measure (range) 


7 


47 



In the above table, Student B exhibits inconsistent performance because there is far 
more variability in his/her standardised scores, illustrated by an inconsistency measure 
of 47 (the difference between 119 and 72). 

For the main sample, the inconsistency measure varied from a value of 0 to 80, with a 
mean value of 15. In order to explore this inconsistency in performance further, it was 
analysed with respect to background factors. To do this in a valid way it was 
necessary to consider all background characteristics together, rather than one at a 
time. (As some of these factors are strongly related to each other considering one at a 
time may give misleading results.) 

A regression 5 model was fitted, to look at the relationship between the inconsistency 
measure and background characteristics - sex, ethnicity, eligibility for free school 



? Regression analysis (linear): this is a technique for finding a straight-line relationship which allows us 
to predict the values of some measure of interest (‘dependent variable’) given the values of one or more 
related measures. For example, we may wish to predict schools’ GCSE performance given some 
background factors, such as free school meals and school size (these are sometimes called ‘independent 
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meals, English as an additional language and institution type. Results of this 
regression are shown in Table 6.2, which shows the coefficients for background 
characteristics that were significant at the 5 per cent level. The coefficients give the 
expected change in the inconsistency measure from the ‘default’ category to the given 
category, where the ‘defaults’ were taken as male, white, not FSM, not EAL, in a 
comprehensive school. In other words a negative coefficient indicates less 
inconsistency in performance than the default category and a positive coefficient 
indicates greater inconsistency than the default. 



Table 6.2: Significant coefficients of regression model to predict 
inconsistency of performance 



Background Characteristic (v. default) 


Coefficient 


Female (v. male) 


-0.5 


FSM (v. not FSM) 


+1.6 


Chinese (v. white) 


+3.6 


Ethnicity unknown (v. white) 


+1.3 


EAF 


+1.8 


Grammar school (v. comprehensive) 


-1.9 


Independent school (v. comprehensive) 


-1.7 


FE college (v. comprehensive) 


+3.3 



From the above table it appears that female students tend to be more consistent (less 
inconsistent) in their performance than male students and that students in grammar 
and independent schools tend to be more consistent in their performance than 
comprehensive school students. Conversely, Chinese students, students eligible for 
free school meals, those learning English as an additional language and students from 
FE colleges tend to be less consistent (more inconsistent) than the default categories. 
The inconsistency analysis showed that such students were more likely to have 
inconsistencies between the SAT®, A levels and GCSEs but did not show that these 
were systematically in any particular direction. A similar model using an 
inconsistency measure related to standard deviation rather than range produced the 
same significant coefficients. 



variables’). When there are several background factors used, the technique is called multiple linear 
regression. 
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For the second part of this analysis the focus was on individuals who had better than 
expected SAT® scores, given their performance at A level (and at GCSE) - in order 
to examine the characteristics of such students. To explore this, another regression 
model was run, this time with average SAT® score (across reading, mathematics and 
writing) as the outcome, and total A level score as one of the background factors, as 
well as all the other factors used in the previous regression. The idea here was that, 
having controlled for A level score, the other coefficients would indicate categories of 
students who perform better or worse on the SAT® than might be expected. Table 6.3 
contains the coefficients of two analyses, one controlling for A level total score and 
one controlling for both A level total score and mean GCSE score. 



Table 6.3: Significant coefficients of regression model to predict 
higher or lower than expected SAT® performance 



Background Characteristic 
(v. default) 


Coefficient 
(A level only) 


Coefficient 

(A level + mean GCSE) 


A level (per point achieved ) 


+0.21 


+0.08 


GCSE (per point achieved ) 


- 


+8 


Female (v. male) 


-17 


-22 


Asian (v. white) 


-27 


-26 


Black (v. white) 


-22 


- 


Chinese (v. white) 


-47 


-33 


Ethnicity unknown (v. white) 


-14 


-10 


SEN (v. no SEN) 


-20 


-10 


EAL (v. no EAL) 


-13 


-9 


Grammar school (v. comprehensive) 


+21 


+9 


Independent school (v. comprehensive) 


+30 


- 


FE college (v. comprehensive) 


-13 


- 



For the model predicting overall SAT® score, controlling just for A level results 
without GCSE included, the explanatory power was such that 44.7 per cent of the 
variance in the SAT® outcome was explained (R“ = 0.447). When GCSE average 
score was also included in the model 49.5 per cent of the variance was explained (R~ 
= 0.495). 
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Results were largely similar whether GCSE was included in the analysis or not. 
Females, ethnic minorities and those with special educational needs or English as an 
additional language appeared to perform less well on the SAT® than expected, 
compared in each case with the default group, and students in grammar schools 
seemed to perform better on the SAT® than might be expected from other results. 
Students in independent schools also did better than expected when A level results 
were taken into account and Black students and students in FE colleges worse than 
expected. In each case they neither did better nor worse than expected when prior 
attainment at GCSE was also taken into account. 

Care must be taken in interpreting these results because of the ceiling effect of A 
levels. Under the current system a certain number of points are allocated for each 
grade rather than using raw examination marks. Although the range of marks within 
the A grade band may vary widely, all students achieving an A grade are awarded the 
same number of points. Therefore, one possible explanation for the findings relating 
to grammar schools and independent schools is that students appear to do better than 
expected on the SAT® because their A level attainment is capped. The effect is much 
reduced once GCSEs - where less of a ceiling effect occurs - are taken into account. 

The free school meals eligibility indicator is often viewed as a proxy for 
disadvantaged circumstances. However, eligibility for free school meals was not 
significant in either of these analyses when attempting to predict average SAT® 
performance, i.e. there was no tendency for students eligible for free school meals to 
perform better (or worse) in the SAT® than in the other two attainment measures. 
However, when the regression model controlling for both A level total score and 
mean GCSE score was run for each of the three main SAT® components separately, 
students known to be eligible for free school meals did better than would be expected 
on the SAT® reading component compared to students that were not eligible. In other 
words students eligible for free school meals tended to achieve higher reading scores 
than non-eligible students with similar attainment at KS4 and KS5. However, in both 
regression analysis models to predict higher or lower performance in the SAT® 
mathematics component, eligible students did worse than expected. 

Generally, the results of the analyses of the separate components were similar to those 
for the SAT® as a whole. The only other exceptions were that Chinese students did 
better than might be expected in mathematics and female students did better in 
writing. 



41 




6.1 Conclusions 



Without higher educational outcomes it is not possible as yet to answer the question 
as to whether the SAT® can identify students with the potential to benefit from HE 
whose ability is not adequately reflected in their A levels because of their 
disadvantaged circumstances. 

The current analysis has revealed that: 

• Chinese students, students from FE colleges and students eligible for free school 
meals or learning English as an additional language tend to show greater 
inconsistencies in performance between the SAT®, A levels and GCSEs 
compared to default groups (white, students from comprehensive schools, etc) but 
these are not systematically in any particular direction. 

• Taking into account their KS4 and KS5 attainment, female students, ethnic 
minorities and those with special educational needs or learning English as an 
additional language appear to perform less well on the SAT® than expected and 
students in grammar schools seem to perform better on the SAT® than might be 
expected. 

These results should be treated with caution until more sensitive analyses have been 
conducted. One of the problems with using categories such as eligibility for free 
school meals in the analysis was that only a very few students were known to be 
eligible and, for many students, information as to eligibility was missing. The 
intention is now to carry out more complex statistical modelling of this and additional 
data. This more detailed examination of social, educational and economic background 
variables will utilise data on the attainment of the institutions attended, census data 
relating to the home postcodes of students in the sample and, where available, 
information supplied by the students themselves (e.g. parental education). Although 
no final conclusions regarding this research question can be reached until we have HE 
outcomes for students in the sample, this further analysis may reveal more about the 
performance of students across the SAT®, A levels and GCSEs according to more 
sensitive measures of economic and educational disadvantage. 
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7 High-achieving students: 

Can the SAT® distinguish helpfully between the 
most able applicants who get straight As at A 
level? 

This section of the report presents the findings relating to those students who achieved 
at least three A grades at A level. Three A grades was chosen as a particular 
benchmark as being the level now required to gain entry to some leading UK 
universities. There were 1402 such individuals, 17 per cent of the main sample, 
broken down as follows: 

Table 7.1: Frequencies of students with three or more grade As at A 
level 



Number of grade As at A 
level 


Frequency 


Per cent 


3 


835 


59.6 


4 


437 


31.2 


5+ 


130 


9.3 


Total 


1402 


100.0 



Due to rounding, percentages may not sum to 100. 



A comparison of the background characteristics of this group of students compared to 
the main sample is given in Table 7.2. 

In terms of their background characteristics, a higher percentage of female students 
achieved three or more As at A level compared to the percentage of female students in 
the sample as a whole. Only one per cent of those students who achieved three As at 
A level were eligible for free school meals compared to approximately three percent 
in the sample as a whole. Hence, there were only nine students out of a total of 243 
students eligible for free school meals in the three As group. Chinese students were 
the only ethnic group that proportionally increased in the three As group, along with 
those students whose ethnic origin was missing (generally students from independent 
schools). Finally higher percentages of grammar and independent school students 
were found in this category than in the sample as a whole and a much lower 
proportion of students from comprehensive schools. 
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Table 7.2: Background characteristics of the sub-sample of students 
with three or more grade As at A level 







Main 

per cent 
(n = 8041) 


Three As 

per cent 
(n = 1402) 


Sex 


Male 


46 


39 




Female 


54 


61 


Ethnicity 


Asian or Asian British 


8 


7 




Black or Black British 


1 


1 




Chinese 


1 


2 




Mixed 


2 


1 




White 


77 


70 




Other 


1 


1 




Missing 


8 


18 




No 


70 


48 


EAL 


Yes 


7 


4 




Missing 


23 


48 


SEN 


No provision 


92 


89 




School Action (A) 


2 


2 




School Action Plus (P) 


0 


1 




Statement of SEN (S) 


0 


0 




Missing 


5 


9 


FSM eligibility 


No 


97 


99 




Yes 


3 


1 




Comprehensive schools 


52 


20 


Post- 16 institution 


Grammar schools 


21 


29 


type 


Independent schools 


22 


48 




FE / sixth form colleges 


4 


3 


Total 




100 


100 



Students who had gained three or more grade As at A level achieved significantly 
higher scores than the rest of the sample on all subtests of the SAT®. As one might 
expect, they also differed significantly from the rest of the sample in terms of their 
prior attainment (GCSE total point score). 
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Correlations between the GCSE and A level scores and each of the SAT® scores for 
these students were also examined. The correlations between the different measures 
for this sub-sample of students were lower than those given in Table 5.9 for the main 
sample due to the restricted range of GCSE and A level scores within this group. 
Apart from within test correlations, the strongest correlation was between the SAT® 
total score and the average GSCE points score (0.44). The correlations between the 
SAT® total score and the GCSE and A level total points scores were 0.32 and 0.31 
respectively. These were all higher correlations than that between the A level and 
GSCE total point scores (0.25). 

An important aspect of the analysis was to consider the extent to which students who 
achieved three or more A grades at A level achieved similar scores on the SAT® test, 
i.e. was the SAT® able to differentiate between them or were their scores clustered 
together. For this part of the analysis a further group of 838 students who achieved 
two A grades were also included. The ranges of scores on the different SAT® 
components for these students, grouped by the number of A grades at A level 
achieved, are presented in Figures 7.1 to 7. 3. Table 7.3 summarises the median, 
minimum and maximum SAT® scores for each group of students. 



Table 7.3: SAT® medians and score ranges by A grades at A level 





Critical Reading 


Mathematics 


Writing 


2 A grades 


560 


540 


550 


280-800 


250-800 


370-710 


3 A grades 


600 


585 


570 


320-800 


310-800 


370-760 


4 A grades 


630 


640 


610 


420-800 


400-800 


430-770 


5+ A grades 


650 


670 


620 


510-800 


490-800 


460-780 



score ranges exclude outliers and extreme values 
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Figure 7.1: SAT® reading score by number of A grades at A level 6 



SAT® reading score 




6 Box and whisker plots give a visual display of summary statistics. The median is represented by the 
black horizontal line. The box denotes the upper and lower quartiles (75 th and 25 th percentiles). The 
vertical lines (whiskers) extend to minimum and maximum values with outliers (values between 1.5and 
3 box lengths from the upper or lower edge of the box) and extreme values (values more than 3 box 
lengths from the upper or lower edge of the box) displayed beyond. 
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Figure 7.2: SAT® mathematics score by number of A grades at A 
level 

SAT® maths score 




No. of A grades at A level 



Figure 7.3: SAT® writing score by number of A grades at A level 
SAT® writing score 




No. of A grades at A level 



47 











As can be seen from the figures above there were wide variations in SAT® scores 
amongst these students, particularly those with two or three A grades at A level. 
Variation was greatest in the reading and mathematics scores. This suggests that an 
aptitude test such as the SAT® might offer scope for distinguishing between 
candidates with similarly high A level grades, particularly amongst students who are 
prevented by their schools or colleges from attempting more than three full A level 
courses. 

However, the usefulness of using the SAT® in addition to A levels to select 
candidates for admission to HE courses cannot be established until the relationship 
between SAT® scores and degree outcomes is known. It is hoped that this research 
study will be able to shed some light on this issue in due course. It is also possible that 
differentiation might be achieved in other ways, for example by using the raw marks 
achieved by students in their A level courses rather than grades. 

In order to investigate the characteristics of these high-achieving A level students 
some analysis was carried out using logistic regression. This analysis included both 
the separate SAT® subject scores and the background characteristics investigated in 
the previous sections of this report. 

It was found that, of the different SAT® components, the best predictor of whether a 
student had achieved three A grades was the mathematics score - the strength of the 
relationship was about twice that of any of the other SAT® scores. Controlling for 
SAT® score, the other factors which appeared to increase the probability of obtaining 
three or more A grades were: 

• being female 

• being of Asian, Chinese origin or unknown ethnic group (compared to white 
students) 

• attending an independent school (strongest), or grammar or FE college 
(compared to attending a comprehensive school). 

Of these factors, attendance at an independent school was the strongest. 

The reason that the mathematics score was more useful in predicting whether a 
student obtained three or more A grades requires further more detailed examination. 
Future analyses will consider sub-samples of students grouped by subjects studied at 
A level in order to ascertain whether the relationship between SAT® scores and A 
levels is reasonably consistent across different groups or whether some SAT® 
components may predict ‘success’ in some subjects but not others. 

One outcome from the regression model was a table showing the number of students 
that were correctly or incorrectly predicted to get three or more As at A level from 
their SAT® scores and background characteristics. Of the 8041 students 87 per cent 
were correctly placed; 95 per cent of the students who did not achieve three or more A 
grades and 45 per cent of those who did. Of the 1402 students who achieved three or 
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more As at A level, 775 would not have been predicted to have done so. There were a 
further 303 students who would have been predicted to achieve three or more As at A 
level that did not in fact do so. The aim of the study is not to see if the SAT® can 
predict A level attainment but to ascertain whether the addition of the SAT® can 
better predict higher education outcomes than A levels alone. However, this is an 
interim exercise to show that the SAT and A levels are measuring similar but not 
identical constructs. If the model was able to correctly predict almost everyone, it 
would suggest that the SAT® and A levels are measuring the same thing. Conversely, 
if the prediction levels were too low, there would be a concern that the SAT® was not 
measuring educational aptitude. 

A further regression analysis was carried out using average GCSE score, as well as 
SAT® scores and background characteristics, to predict three or more A grades at A 
level. In this case the most important factor in predicting three grade As was average 
attainment at GCSE. (A similar model substituting total points for average GCSE 
score showed that the GCSE total points figure was much less useful as a predictor.) 
Although much less important than prior attainment in predicting three or more A 
grades, the reading, mathematics and essay components of the SAT® were also 
significant factors. Again this may suggest that they are measuring a slightly different 
construct than GCSEs. Controlling for SAT® scores and prior attainment, the 
significant background variables that increased the probability of obtaining three A 
grades were: 

• attending an FE college (compared to a comprehensive) 

• eligibility for free school meals unknown (probably students mainly at 
independent schools) 

• learning English as an additional language (at this stage of their education 
their fluency in English is not likely to be a significant barrier). 

To look at this group of students in a slightly different way the following questions 
were examined: 

• What were the background characteristics of students that achieved high 
scores on the SAT® but were not part of the three A grades A level group? 

• What were the background characteristics of students that did not achieve high 
scores on the SAT® but were part of the three A grades A level group? 

To explore this issue all students in the main sample were split into three groups 
according to their total SAT® score; the higher group representing the top five per 
cent of scores and the high group the next ten per cent of scores. Each of these groups 
was further split into two according to whether students had achieved three or more A 
grades at A level as shown in Table 7.4. 
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Table 7.4: SAT® scores by number of A grades at A level 



Total SAT® score 


A levels with 3+ A grades 
No Yes 


Total 


Low (600-1790) 


6144 


(90%) 


706 


(10%) 


6850 


High (1800-1950) 


404 


(49%) 


414 


(51%) 


818 


Very high (1960+) 


91 


(24%) 


282 


(76%) 


373 


Total 


6639 




1402 




8041 



This found 495 students with a high or very high SAT® score who did not achieve 
three or more As at A level. There were also 706 students that did not do so well on 
the SAT® who did achieve three or more A grades. The background characteristics of 
these two groups were then examined. 

A higher proportion of male students compared to females were in the high SAT® 
performance categories, but achieved less than 3 As at A level. In particular 65 of the 
91 students with very high SAT® scores were male. Conversely more females than 
males achieved three A grades in the ‘low’ SAT® category. It is interesting to note 
that some male students did extremely well on what was for them a low-stakes test, 
even though they did not subsequently achieve three A grades at A level. Whether this 
is due to the content of the SAT® or the nature of the assessment (examination only 
and mainly multiple-choice) is not known. The relevance of this finding and whether 
the additional information offered by the SAT® would be useful to HE admissions 
staff will depend on whether the combination of these scores will better predict 
university outcomes than A levels alone. This will not be known until data as to the 
HE attainment of these students becomes available. 

Compared to comprehensive schools, more students from independent and grammar 
schools in the low SAT® group achieved three As at A level. There were also more 
students from these schools in the groups with both high SAT® scores and grade As 
at A level. Within the main sample as a whole, students in grammar schools were 
more likely to obtain three or more A grades at A level compared to comprehensive 
students (as reported on page 45). However, amongst students with high or very high 
SAT® scores, the percentage of grammar school students who did not achieve three A 
grades was slightly higher than would be expected. This tendency was true both for 
male and female students but more noticeably amongst male students - although the 
numbers in these sub-groups were fairly small. 

Students for whom information was missing concerning English as an additional 
language and eligibility for free school meals (largely students from independent 
schools) were more likely to achieve A grades compared to all other groups of 
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students. Students learning English as an additional language and students eligible for 
free school meals were more likely to achieve low outcomes on both measures. 

7.1 Conclusions 

Amongst a sub-sample of students with three or more A grades at A level there were 
wide variations in SAT® scores. This suggests that the SAT® might offer scope for 
distinguishing between candidates with similarly high A level grades. 

Although there was considerable overlap between attainment at A level and SAT® 
scores, a substantial number of students did better on one measure than the other. 
Some students, who might have failed to gain admission to the most prestigious 
universities on the basis of their A level grades, nevertheless achieved scores in the 
top 15 per cent or even top five per cent of SAT® scores. On the basis of the current 
analysis the sub-group of potentially able students who might benefit most from the 
SAT® would be male students. As discussed at the end of section 6, other sub-groups 
who might potentially benefit may be identified using more complex statistical 
modelling of the data. 

In due course when HE outcomes become available, another important sub-group to 
consider will be those whose attainment at both GCSE and A level was high but who 
performed poorly on the SAT®. 

If the SAT® alongside A levels can be shown to better predict university outcomes, 
the SAT® might offer a means of enabling differentiation between students with the 
same grades, as well as an alternative measure of ability for those students who for 
whatever reason fail to show their full potential by means of A levels. 
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8 Questionnaire findings 

Two questionnaire surveys have been conducted to gather qualitative data about the 
student sample. The background characteristics of the survey sub-samples are 
described in section 4.4 and copies of the questionnaires annotated with students’ 
responses are given in Appendices 1 and 2. This section presents a summary of the 
key findings from the two surveys. 

8.1 Spring 2006 questionnaire 

As reported in section 3.3, over 6800 students took part in the spring 2006 
questionnaire survey. As well as providing some background details about their home 
and family circumstances, the questionnaire asked them about their experiences of 
school or college in years 12 and 13, their immediate plans after A levels and their 
views of higher education. 

8.1.1 Future Plans 

The questionnaire contained questions about their plans after A levels and their 
aspirations in terms of future careers. 

• Ninety-two per cent of the students who completed the survey indicated that they 
would definitely or probably go to university in either 2006 or 2007. Seventy-four 
per cent expected to commence a degree course in 2006, with a further four per 
cent on a foundation course and 17 per cent taking a gap year. 

• Students with a parent who had been to university were more likely to say that 
they definitely intended to go to university than students with a parent who had left 
school at age 18 and they, in turn, were more likely to say they definitely intended 
to go to university than students with a parent who had left school at age 15 or 16. 

• Regarding sources of information about higher education, over 90 per cent had 
looked at university produced information (website, prospectus or CD) and over 
80 per cent had been to a university open day. 

• Most students indicated that the vast majority of students in their year group were 
also planning to go to university. However, over 800 students estimated that about 
half or less of their year group was intending to go to university. 

• About 60 per cent of the students were absolutely certain ox fairly certain what job 
they would like to do in the future. The most popular careers were medicine (15 
per cent); engineering, science and IT (14 per cent); health and social welfare, 
such as social work, nursing and the police force (12 per cent); and teaching (11 
per cent). 

At the stage at which these responses were given these career choices and the 
associated degree paths represented students’ aspirations. In the next phase of the 
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project actual destinations will be examined and the extent to which students’ 
aspirations have been achieved will be explored. 

8.1.2 Views on university 

All students were asked to give their views of university, irrespective of whether they 
intended to go or not. Students gave views by indicating agreement with a number of 
statements on a five point scale ( strongly agree, agree, not sure, disagree, or strongly 
disagree). 

• Over 80 per cent of students indicated that they would enjoy studying for a degree, 
that their parents / carers would be pleased if they went to university and that most 
of their friends were going. 

• Twenty-three per cent of students were not sure whether their results would be 
good enough to go to university, but only three per cent did not think that they 
would be good enough. 

• Thirteen per cent of the questionnaire sample agreed with the statement I don ’t 
think I can afford to go to university. 

• The most popular reasons for applying to university were interesting subjects / 
courses (86 per cent), social life (65 per cent), more interesting jobs / careers (65 
per cent), requirement of chosen career (63 per cent), earnings potential (58 per 
cent). 

• Over 120 institutions were cited as the preferred choice for higher education. The 
five institutions most frequently cited were the universities of Manchester, Leeds, 
Birmingham, Nottingham and Cambridge. 

• The most important reasons for selecting a particular university to study at were: 
good academic reputation (75 per cent), facilities (74 per cent), location (67 per 
cent), social life (62 per cent), graduate employment prospects (49 per cent), best 
available course (48 per cent). 

8.1.3 Views on school 

Students also gave views of their school or college and their current studies, again by 
indicating agreement with a number of statements on five point scales. Generally such 
opinions were positive. 

• Most students (79 per cent) found the work at school or college interesting and 
very few (4 per cent) regretted staying on after year 11. 



7 In this summary the percentage agreement / disagreement figure represents those students who agreed 
and strongly agreed with a statement or who disagreed and strongly disagreed respectively. 
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• Approximately 90 per cent of students agreed with the statement: My studies at 
school / college will help me in the future, suggesting that they considered their 
courses to be relevant to their intended careers or could see the importance of the 
transferable skills they were acquiring. However, only 50 per cent of students felt 
that they had been given good advice when choosing their A levels. 

• Just over 50 per cent of students did not think that their AS level results reflected 
their true ability. The most popular reasons given were not doing enough work (69 
per cent), personal circumstances (25 per cent), not enjoying the work (24 per 
cent) and the quality of teaching (19 per cent). 

• Most students were happy with the quality of the teaching they received. Less than 
eight per cent thought the teaching was poor and only a small proportion, just over 
six per cent, thought that their school or college was not a good place to study. 

8.1.4 Careers advice 

Students were also questioned about the advice given to them by their school or 
college to help them make the right decisions for the future. 

• Over 40 per cent of students rated the careers advice they had been given as 
helpful, but over 20 per cent of students had not found the advice helpful or 
claimed not to have received any. 

The students were asked specifically about how well their last two years at school had 
prepared them for the future. Each aspect was rated on a four point scale from very 
well to not at all well. 

• Over 90 per cent indicated that their education in year 12 and year 13 had 
equipped them with useful skills and knowledge. 

• Over 80 per cent had been given helpful information about university but less than 
50 per cent felt they had been given good information about jobs and training or 
encouraged to look into vocational training. 

• Although the majority of students thought their school had prepared them for 
studying at university, a sizeable proportion thought they were not very well 
prepared (24 per cent) or not at cdl well prepared (4 per cent). 

8.1.5 Students not intending to go to university 

Approximately 500 students (8 per cent of the questionnaire sample) indicated they 
would probably not or definitely not go to university. The most common reasons were 
wishing to pursue a job or career that did not require a degree (38 per cent), worries 
about debt and finance (38 per cent) and having no desire to study further (32 per 
cent). 
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An analysis of the students who indicated that they were unlikely to go to university 
suggested that very few students in the higher end of the ability range were allowing 
worries about debt to prevent them from planning to go on to higher education. 
Although the number of students involved is very small, this issue will be examined 
again in the 2008 report with actual post A level destinations. 

8.1.6 Opinions of the SAT Reasoning Test™ 

Students were asked their opinion as to whether the SAT® was likely to better reflect 
their true ability than their AS results. Over half of those who responded indicated that 
it would not, approximately one third were unsure and a small percentage thought that 
it would. 

Approximately half of the students who completed a questionnaire took the 
opportunity to write an open comment on the SAT®. Although four per cent of such 
responses were positive, the most common comments stated that the SAT® was 
repetitive, boring or long (22 per cent) or that it was irrelevant to their current studies 
(11 per cent). 

8.2 Autumn 2006 questionnaire 

A smaller number of students (approximately 3300) took part in the autumn 
questionnaire survey, and of these 3177 responses were used in the analysis (as part of 
the main sample). This second questionnaire filtered participants into answering 
particular questions depending on their intentions with regard to higher education: 
entering HE in 2006; taking a gap year with a deferred place in 2007; or no intentions 
/ immediate plans for HE. The following summary groups the findings accordingly. 

8.2.1 HE entrants in 2006 

Over three quarters of the autumn survey sample (2455 students) indicated they had 
started a course at an HEI in autumn 2006. 

• Eighty- six per cent of these students had obtained their first choice of HEI, six per 
cent had gained their second choice and eight per cent had obtained a place 
through the clearing system. 

• As reported in section 4.2.2, approximately 60 per cent of these 2455 students had 
commenced a course of three years’ duration, and a further 30 per cent were on a 
four year course. 

• Over 80 per cent were planning to live in a hall of residence during their first year. 

• Over 95 per cent were fairly confident or very confident of their ability to complete 
the course. 

• When asked about how they were planning to cover their total expenses during the 
year (including course fees, accommodation, food, etc), over 90 per cent indicated 
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they would have to take out some form of loan that would need to be repaid. 
Almost half of all respondents indicated that this would be in excess of £4000. 



8.2.2 Gap year students 

A small number of students in the survey sub-sample (285) planned to take a gap year 
and had a guaranteed or deferred place at an HEI for autumn 2007. 

• Students were asked to indicate which activities they were planning to do during 
their gap year from a range of seven options. The two most popular options were 
paid employment in the UK and travel. 

• When asked to indicate their main reason for taking a gap year, the most frequent 
response was to have a break from studying, closely followed by wanting to travel. 

8.2.3 No immediate plans for HE 

Just over 400 respondents had no immediate plans to enter higher education. Of these 
190 were in some form of employment and 108 were in full or part-time education. 

• Asked how likely it was that they would apply for a place at university in the 
future (on a four point scale from definitely will to probably not ) 204 indicated that 
they definitely will and a further 77 that they probably will. 

• The remaining respondents (approximately 100) were asked to indicate their 
reasons for not going to university from a list of options and to rate the importance 
of each reason in their decision. The reasons rated as being very important or 
somewhat important by a majority were (in order of frequency): worried about 
getting into debt; not wishing to study further; not being able to afford to go; not 
needing a degree for their job; and happy in the job they are doing. 

In subsequent phases of the research, it is hoped to relate the information collected 
from these surveys to the university destinations and degree outcomes of students in 
the main sample. 
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9 Future phases of the research 

As mentioned at various points in this report there are still further analyses to be 
carried out with the current data, namely: 

• further exploration of the relationships between SAT® scores and attainment 
in particular subjects, or groups of subjects, at GCSE and A level 

• more complex modelling of the background data of students in the sample to 
create more sensitive measures of economic and educational disadvantage (in 
order to better answer the question as to whether the SAT® can identify 
students with the potential to benefit from HE whose ability is not adequately 
reflected in their A levels because of their disadvantaged circumstances). 

The results of these further analyses will be presented in a second published report in 
2008. This report will also examine the destinations of students in the sample, relating 
such information (where possible) to previously-collected data on both background 
and attainment. HESA information on the university destinations of students in the 
main sample will be provided by HEFCE and some analysis of this will be carried 
out, to see if there are factors other than A level attainment which seem to impinge on 
the chances of acceptance into particular kinds of institution. This report will be 
published in May 2008, provided the destination data is available by January 2008 as 
scheduled. 

The final report in 2010 will follow the collection of degree outcomes from HEFCE. 
It will attempt to relate these to the SAT® scores and the A-level outcomes, adjusting 
as far as possible for the loss of those not selected for university places. A multilevel 
model will be set up to examine whether the initial aptitude test results gave 
significantly improved predictions of degree outcomes. The analyses will explore the 
effects of different types of school and university. Separate analyses for some popular 
subjects might be possible, as well as analyses within universities. (Such analyses will 
of course be reported in anonymised form.) This report could be completed by April 
2010, depending on the availability of the degree outcomes data. Further analyses 
could include those students graduating after 2009 subject to an extension to the 
existing contract. 

Once completed, this research will enable some important conclusions to be made 
about the use of the SAT® or similar aptitude test in university admissions. Each of 
the research questions listed in section 2.2 requires a study of the relationships 
between A levels (and GCSEs) and university degree results, on the one hand, and the 
SAT® and university degree results, on the other. The success of the SAT® in 
fulfilling the specified purpose will be demonstrated if it can be shown that the 
combination of the SAT® and A levels provides a better prediction of degree success 
than A levels alone. In addressing the question about students in disadvantaged 
circumstances, such a pattern of correlations will provide validation evidence for the 
SAT® in identifying those with potential - attested by their eventual degree results - 
not recognised by A levels. In addressing the question regarding the most able HE 



57 




candidates, the SAT® will be valid for this purpose if it provides additional 
discrimination amongst the highest attaining students that overcomes the ceiling effect 
of A levels. 
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Appendix 1: Spring 2006 Student Questionnaire 

The data provided is based on the responses of up to 6825 
students who took part in the spring 2006 student survey. 
Unless otherwise stated, the figures represent the percentage 
responses for each question or item. Figures may not sum to 
100 either due to rounding to the nearest integer or because 
more than one answer could be given. 




your individual experience 
of higher education 



The NFER is currently conducting research on the use of an aptitude test in university entrance. 
You may remember taking the SAT test earlier this academic year. At the beginning of the 
summer term you will receive your results, an explanation of what these mean and a certificate 
to add to your personal portfolio. As part of this on-going research, we would like to find out 
about students’ opinions of school / college and higher education. We would therefore be very 
grateful if you would complete this questionnaire. For most questions, please just / the box 
which is closest to what you think. All your answers will be treated in complete confidence. 

The questionnaire should take about 20 minutes to complete. When you have finished, please 
put it into the envelope provided and return it to your teacher / administrator. 

Thank you for your help. 

Please check the following information (as supplied on your agreement form) and amend as 
necessary. These details will be used to contact you in the future about your progress but will not be 
passed on to anyone unconnected with this research. 



First name: 




Last name: 




Gender: 


Date of birth (dd/mm/yy): 


Flome address: 




Town: 


County: 


Postcode: 




e-mail: 




Mobile phone number: 





0381/82 (VAU) © 2006 National Foundation for Educational Research 
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Please note: 

You can study for higher education qualifications, such as degrees and higher education diplomas, 
in many different places, for example universities, other higher education institutions (HEI’s) and 
some further education colleges. For simplicity, in this questionnaire we refer only to universities 
and degrees. However, university includes the range of different places offering these courses and 
degree means any higher education qualification. 



What next? 



1 What are you intending to do in the 12 months following your A level results? 
(Please / all that apply.) 



Complete a Foundation Course / Access Course 
Study for a degree at a university 
Take a year out (gap year) 

Do more A levels 

Do a vocational course (e.g. BTEC National Certificate) 

Go into any kind of paid work 

Go into paid work with good career prospects 

Do a Modern Apprenticeship / Advanced Apprenticeship 

Do voluntary work 

Look after the home / family 

Don’t know 

Something else (please say what) 
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e.g. travel 



2a Do you have a specific job or career you would like to do in the future? 
(Please / one box only.) 



Yes, absolutely certain 

2l]i 



Yes, fairly certain 

40l 2 



Unsure 

03 



No, no idea at all 

04 



2b What job or career is that? 



Most frequent responses: medicine; engineering; health and social welfare 
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We are interested in your views of university, whether or not you plan to go. Please answer the 
following questions. 



3 Please tell us if you agree or disagree with the following statements. 
(Please / one box in each row.) 



Strongly Agree Not Disagree Strongly 

agree sure disagree 


1 think 1 would enjoy studying for a degree 


42 




48 




8 




2 


0 


1 think my exam results will be good enough 
to get me into university 


23 




52 




23 




2 


0 


1 think 1 would find it easy to fit in at a university 


22 




@ 




19 




2 


0 


The work at university would be too hard for me 


0 


0 


41 




47 


0 


Most of my friends are going to university 


43 




S 




6 




5 


0 


1 would want to live at home if 1 went to 
university 


6 


0 


13 




28 




46 




My parents / carers would be very pleased 
if 1 went to university 


58 




35 


0 


0 


□ 


My friends would think 1 was a snob 
if 1 went to university 


0 


0 


3 




25 




70 




1 think my parents / carers would put me under 
too much pressure to do well at university 


2 




22 




43 




23 




The new system of financial support for students 
makes it more likely that 1 shall go to university 


5 




18 




37 




27 




13 




The introduction of higher fees makes it less likely 
that 1 shall go to university 


6 




15 




27 




40 




12 




1 don’t think 1 can afford to go to university 


3 




24 




44 




19 





3 



4 



Please tell us if you agree or disagree with the following comments. 
(Please / one box in each row.) 



Life at university is just like school / college 

Most people who go to university end up in debt 

People who go to university have to be very clever 

Students at university never do any real work 

People with degrees get better paid jobs 

The only way most people can afford to live at 
university is to have a part-time job 

The cost of university will be worth it in the 
long run 

I think you can be successful without a degree 

Studying at university will be more interesting 
than school / college 



Strongly 


Agree 


Not 


Disagree Strongly 


agree 

i 


2 


sure 

3 


4 


disagree 
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5 For which of the following reasons might you go to university? 
(Please / all that apply.) 



The subject / course sounds interesting 

A degree is essential for the career / career path I have chosen 
Graduates earn more money 
A degree will help me get a more interesting job 
I am attracted by the social life 

I am being encouraged to go by people who are close to me 
I want to get away from home 

It will give me time to think about what I want to do next 
Other (please say what) 
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e.g. personal development 



6a 



Do you think you will go to university in either 2006 or 2007? (Please V one box only.) 
Definitely Probably Probably not Definitely not 
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If you do not intend to go to university in 2006 or 2007, please go to Question 7a. 






6b If you have already applied to university, please tell us your UCAS number, if known. 



6c If you have chosen the subject you wish to study, what reasons influenced your choice? 
(Please / all that apply.) 



It is the subject that 1 am best at in school 


40 


1 


1 find the subject easy 


14 


It is the subject that 1 enjoy most at school 


61 


2 


It is easy to get on a course 


3 


The subject is essential for my intended 
future career 


52 


3 


Advice from teachers 


14 








Advice from parents 


14 


Other (please say what) 


5 


4 







e.g. interest in subject 



6d At this stage, which university or other institution would be your preferred choice for higher 
education? 



6e Why is this institution your preferred choice? (Please / all that apply.) 



It has a good academic reputation 

The course is the best available in my chosen subject 

I want to stay close to home 

I like the city / town where it is located 

It has good facilities 

It offers a good social life 

To be with my friends / partner 

Graduates from there have good employment prospects 
It is easy to get into 

Other people in my family have been there 
Other (please say what) 
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e.g. specific comment about course 



Now go to Question 8a. 



7 a If you are not intending to go to university in 2006 or 2007, would you consider university in 

the future? (Please S one box only.) 



Yes 

39 



No 

17 



Don’t know 

43 



7b If you do not intend to go to university, please indicate which of the following reasons 
have influenced your decision. (Please S all that apply.) 



I am worried about debt / finance 

I have been unable to obtain an offer of a place so far 

I wish to pursue a job / career that does not require a degree 

I think work at university would be too difficult 

My parents / family do not wish me to go 

I do not wish to move away from family / friends 

My grades / qualifications will not be good enough 

I have a firm offer of a job / employment 

I have no desire to study further 

I have had insufficient information about how to apply 

I have had insufficient information about what courses are available 

I have had insufficient information about what help I could get to pay for university 

It can be difficult to know what it will be like beforehand 

Other (please say what) 
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e.g. undecided 
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Views on school / college 



8a We would like to know how you feel about your studies and school / college at the moment. 
For each statement please tick the box that most closely fits with how you feel. 

(Please / one box on each row.) 



Strongly Agree 

2 



agree 

i 



The work I am doing at school / college is interesting [H 

My studies at school / college will help me in the future [30 

I was given good advice when choosing my A levels 

Teaching standards at this school / college are poor 

Most teachers in my school / college treat the 
students with respect 



0 

0 



18 



I think that this school / college is a good place to study 

Students’ views / opinions are listened to at this 
school / college 

Most teachers at this school / college encourage 
pupils who are good at something 

Most of the time I like being at school / college 

My school / college should concentrate more on 
preparation for tests and exams 

Most teachers at this school / college help students 
who have difficulty with their work 

I usually work hard at school / college 

I regret staying on at school / college after Year 1 1 

Discipline or control problems with other students have 
interfered with my ability to learn at this school / college 



24 



19 



14 



16 



0 



68 



61 



42 



64 



57 



42 



64 



57 



22 



61 



60 



Not Disagree Strongly 
disagree 



sure 

3 



0 

0 

[ 25 ] 



14 



12 



13 



31 



13 

17 

35 



17 



[1®J 

0 



16 



8 



20 



53 



15 



0 



10 



33 



5 

0 



30 

42 



0 

0 

0 

[ 26 ] 

0 

0 

0 

0 

3 

0 

0 

0 

|58l 

30 



8b If you were making the decision about whether or not to stay on at school or college after Y1 1 
again, would you do the same thing over again? (Please / one box only.) 



Definitely Probably Don't know Probably not Definitely not 



63 ! 



22 2 




7 






9a We would like to know what help you think your school / college has given you in preparing for 
your future. (Please / one box on each row.) 



How well do you think your education in Years 12 and 13 has: 


Very 

well 


Quite 

well 


Not 

very well 


Not at all 
well 


equipped you with useful skills and knowledge? 


1 

28 


2 

66 


3 

6 


4 

0 


encouraged you to look into vocational qualifications? 


10 


37 


42 


0 


enabled you to perform to the best of your ability? 


29 


59 


11 


0 


given you helpful information about university? 


32 


50 


16 


0 


prepared you for studying at university? 


18 


54 


24 


0 


given you helpful information about jobs and training? 


8 


38 


42 


0 



9b How many of the students in the same year at your school / college would you say are 
planning to go to university? (Please / one box only.) 



Almost all of Most 

them of them 



About half A few Hardly any 

of them of them of them 






□ 



5 



10 Have you done any of the following things in Year 12 or Year 13? (Please / all that apply.) 



I attended an open day at a university 

I took part in a residential course at a university during term time 
I took part in a summer / winter / Easter school 
I went to an Aimhigher Roadshow 

I had information about higher education from a representative 
of a university who visited my school / college 

I attended a university for an interview 

I have looked at a university website / prospectus / CD 



83j 1 

0, 



07 



11 



How would you rate the careers education and guidance given to you by your school / college 
to help you make the right choices for the future? (Please / one box only.) 

Helpful Not sure Not helpful Haven’t had any 



44 



35 



16 



5 



1 



4 



8 





You and your home 



12a At home do you usually use a language other than English? 



Yes 16 



No 84 



12b If yes, what language is that? 



most frequent responses: Chinese; Gujarati; Punjabi; Urdu 



13a Have you always lived in the UK? 



Yes 87 



No 13 



13b If no, how many years have you lived in the UK? 



mean = 9 



14 Do you live in a rented property or is it owned by someone you live with? (Please / one box only.) 

Owned by parents or someone you live with [88] 1 Prefer not to say [~4~| 3 

Rented [61, Don’t know [~2l . 



15 Who do you live with most of the time? (Please / all that apply.) 



Father 
Mother 

Other adult (e.g. grandmother, cousin) (please say who) [~4] s 



|76ji 

93lo 



Step-father (or mother’s partner) 
Step-mother (or father’s partner) 



04 



e.g. grandparent 



16 How many brothers / sisters (including step brothers / sisters) have lived at home with you at 
some point in the last 5 years (not including yourself)? 



mean = 2 



9 





17a Could you tell us about the education of your mother and father or those person(s) who are 
like a mother or father to you — for example, guardians, step parents, foster parents, etc. 
(Please S all that are true.) 



Did your mother and father: 

finish secondary school (at 15 / 16)? 

study at college or school sixth form (after age 16)? 

study at university or polytechnic / get a degree? 

Don’t know 



Mother / guardian Father / guardian 



59 


1 


57 


49 


2 


45 


35 


3 


41 


8 


4 


8 



17b Do you have a brother or sister who has studied or is studying at university? 



Yes 



37 



No 



63 



18 How much financial support would you receive from your relatives / guardians if you went to 
university? (Please / one box only.) 



Full support, they do not want me to take out a loan 


12 


1 


No support 


a 


Significant support to supplement a loan 


45 


2 


1 don’t know 


s 


Minimal, irregular support 


13 


3 







Questions 19a-19d ask about the parents or step-parents who you live with now. (If you don’t live 
with any parents or step-parents now answer about the parents or step-parents you lived with when 
you were 15 and what they were doing then.) 

19a Are your parents (or step-parents) employed at the moment? 



Father (or step-father) Yes 


92 


No 


8 


Mother (or step-mother) Yes 


80 


No 


20 



19b Please give the occupation of your parent or step-parent who earns the most (e.g. primary 
school teacher, plumber). If he or she is retired or unemployed, give their most recent 
occupation. 



19c Please describe the type of work he / she does (e.g. installs central heating systems). 



10 







19d Does he / she have formal responsibility for supervising the work of other employees? 



Yes 



61 



No 



16 



Not sure 



23 



20 Approximately how often do you do the following in your spare time? 
(Please / one box in each row.) 



Reading not related to course work 
Team sport (e.g. football, hockey) 

Individual sport / fitness (e.g. tennis, yoga) 

Music (e.g. playing an instrument individual 
or in a band / orchestra) 

Other hobbies / out of school clubs 

Part-time job 

Charity or voluntary work 

Look after relatives or dependants 



4+ times 


2-3 times 


Once 


Occasionally 


Never 


per week 


per week 

2 


a week 

3 


4 


5 


24 


24 


15 


30 


0 


9 


16 


14 


29 


0 


13 


26 


22 


27 


12 


y 14 


11 


8 


15 


52 


0 


22 


23 


27 


18 


0 


0 


20 


11 


31 


0 


0 


13 


33 


50 


0 


0 


6 


30 


58 



21 About how many books are there in your home? Do not count newspapers, magazines or 
school books. (Please / one box only.) 



None (0 books) 

Very few (1-10 books) 

Enough to fill one shelf (1 1-50 books) 

Enough to fill one bookcase (51-100 books) 

Enough to fill two bookcases (101-200 books) 

Enough to fill three or more bookcases (more than 200 books) 




_12j 3 

16] 4 

2^5 

47le 



22a Which of the following do you usually prefer when studying? (Please V one box on each row.) 



i tests 


43 


or coursework 


57 




ii team work 


46 


or individual work 


54 




iii independent learning 


38 


or directed learning 


62 





22b Do you set yourself goals when studying and stick to them? (Please / one box only.) 
frequently sometimes rarely 



22 



57 



17 



never 

04 



11 





22c Do you read around subjects (i.e. more than is required to fulfil your coursework)? 
(Please / one box only.) 



frequently 



sometimes 



rarely 



never 



16 



60 



21 



22d To what extent do each of the following motivate you to study? 
(Please / one box in each row.) 



My parents expect me to do well 


a lot 
1 

50 


a little 

2 

44 


not at all 

3 

0 


My school / college expects me to do well 


36 


52 




1 need specific results for what 1 want to do 


73 


22 


0 


1 want to do well 


89 


10 


0 


1 enjoy studying 


29 


53 


S 


Other (please say what) 


43 


22 


36 



e.g. job prospects 



23a Do you feel that your AS results reflect your true ability? 



Yes 



34 



No 



51 



Not sure 



15 



23b If no, which of the following reasons best describe why: (Please / all that apply.) 

5 

6 



e.g. perform poorly on tests 







I didn’t do enough work 
I was very lucky 
The teaching has been poor 
Other (please say what) 



69 



I didn’t enjoy the work 24 

Personal circumstances [25 



19 



13 



12 







24 



Earlier this academic year, you took the SAT test. Do you think the SAT is likely to better reflect 
your true ability than your AS results? (Please S one box only.) 



Definitely 

0 , 



Probably 

~ 8 ~„ 



Don’t know 

34. 



Probably not 

29 , 



Definitely not 

28l 



25 Do you have any comments you wish to make about your experience of taking the SAT test? 



Most frequent responses: 

• repetitive / boring / too long 

• not relevant to current / future studies 

• does not reflect ability 

• limited range of subjects 



26 Do you have any comments you wish to make about university? 

We are interested in your views whether or not you intend to go. 



Most frequent responses: 

• general positive comment 

• comment re future benefits 

• comment re debt / finance 

• comment re personal development 



Thank you very much for your help 

Please put your questionnaire in the envelope provided and return to your administrator 



13 




14 



Appendix 2: Autumn 2006 Student Questionnaire 

The data provided is based on the responses of up to 3177 
students who took part in the autumn 2006 student survey. 
Unless otherwise stated, the figures represent the percentage 
responses for each question or item. Figures may not sum to 
100 either due to rounding to the nearest integer or because 
more than one answer could be given. 




your individual experience 
of higher education 



AUTUMN 2006 



As part of the ongoing UNIQUE research project we would like to find out about your plans 
for higher education or employment, now you have received your A level results. It is very 
important to our research to compare the different experiences of school / college leavers, 
whether or not they intend to enter higher education. 



As a thank-you for your involvement in this research we would like to offer you a chance 
to win £1000 in our free-to-enter prize draw. Five entrants will each win the sum of £1000 
- a total prize fund of £5000. 

Full details and the rules governing the prize draw are given on a separate sheet enclosed with this questionnaire. 



We should be grateful if you would now complete this short questionnaire - it should take no 
longer than 10 minutes of your time. Please note that any information provided by you may be 
used by the NFER and the partners involved in the study for the purposes of this research. All 
your answers will be treated in complete confidence - any report arising from the study will not 
refer to any individuals but will describe the data collected in an aggregated format only. 



Please place your completed questionnaire in the return envelope provided and post it back to us 
no later than 29 September 2006. 

Thank you for your help. 

If you have any questions about this research, please contact: 

David Hereward 

Research Data Services 

NFER 

The Mere 

Upton Park 

Slough SL1 2DQ 

Tel: 01753 637352 

email: d.hereward@nfer.ac.uk 



0383 (VAU) 



SQ 





Please note: 

You can study for higher education qualifications, such as degrees and higher education diplomas, 
in many different places, for example universities, other higher education institutions (HEIs) and some 
further education colleges. For simplicity, in this questionnaire we refer only to universities and 
degrees. However, university includes the range of different places offering these courses and 
degree means any higher education qualification. 



We are interested in the A level grades you have achieved and how these compare with any 
predictions made by your school or college. 



1 For your three best subjects, please circle the grades you were predicted to achieve by your 
school/college at the time of your UCAS application (if known) and your actual A level grades. 
(If the prediction made was at the borderline of two grades, e.g. A/B or B/C please circle both 
grades. Please list full GCE A levels or Advanced VCEs only. ) 

Circle predicted grade/s Circle actual grade 

(e.g. A or A/B) achieved 



Subect 1 


A 


B 


C 


D 


E 


Don’t know 


A 


B 


C 


D 


E 


U 


Subject 2 


A 


B 


C 


D 


E 


Don’t know 


A 


B 


C 


D 


E 


U 


Subject 3 


A 


B 


C 


D 


E 


Don’t know 


A 


B 


C 


D 


E 


U 



2 Will you be starting at a university in September / October this year (2006)? 



Yes 



77 



go to Q3 (n = 2455) 



No 



23 



go to Q11 (n = 713) 



If you are still unsure whether or not you have a place for this year, please wait until you 
are able to answer Question 2 before completing the questionnaire. 



3 If you have a place at university this year (Sept / Oct 2006), was this university your: 
(Please / one box.) 



Firm (first) choice? 
Insurance (second) choice? 



86 



> go to Q6 (n = 2269) 



Place obtained through clearing? 



go to Q4 



(n = 184) 





If you used the clearing system, how many universities did you contact 
(including your final choice) in order to secure a place? (Please / one box.) 



One 



48 



Two 



20 



Three 



11 



Four + 



20 



Don’t know 



0 



If your university place was obtained late or through the clearing system: 

a) Did your results on the SAT® test that you took in year 13 as part of this research project 
encourage you to apply to university? 



Yes 



6 



No, would have gone anyway 



94 



b) Did you inform prospective universities of your SAT® scores? 



Yes Q 



No 



99 



c) If yes, were your SAT® scores helpful in securing your place? 
Yes |jj] No Q Don’t know 0 



What is the duration of the course you are starting in Sept / Oct 2006? (Please / one box.) 
1 year _ 2 _ 2 years 0 3 years 61 _ 4 years 29 More than 4 years 0 



How far is your university from your home? (Please / one box.) 



Less than 10 miles 10 10-49 miles 22 50-100 miles 30 More than 100 miles 38 



Where do you plan to live during the academic year 2006/7? (Please / one box.) 
A hall of residence or other university accommodation (e.g. flat or house) 

A rented flat or house (not owned by university) 

A property owned by you or your family (but not family home) 

With parent(s) or another family member (e.g. sister, brother, grandparent) 



| 82 ji 



^J 2 

B 



1^4 






9 



At this stage, how confident are you in your academic ability to complete the course you have 
chosen? 

(Please / one box.) 



Very confident 


39 


1 


Not very confident 


S3 


Fairly confident 


57 


2 


Not at all confident 


0, 



10 How much of your first year’s total expenses (including course fees, accommodation, food, 
other living expenses, etc) do you expect to cover from each of the following sources? 
(Please / one box in each row.) 



Money from parents / 
family / relatives / spouse 

Money from own resources 
(savings, past earnings, etc) 

Money from (part-time) 
employment during the year 

Money from an employer to 
pay for you to go to university 

Money that does not have to 
be repaid, e.g. grants, bursaries 
(other than from an employer) 

Money that has to be repaid 

(e.g. student loans, bank loans, overdrafts, 

loans from family, friends, etc) 



None 


Less than 
£2000 


£2000 - 
£4000 


£4001 - 
£6000 


£6001+ 


1 


2 


3 


4 


5 


16 


44 


26 


8 


0 


23 


68 


8 


0 


0 


26 


64 


9 


0 


0 


99 


0 


□ 


0 


0 


54 


23 


20 


2 


0 


9 


5 


38 


24 


24 



Now go to Q22 on page 10. 



4 



This section of the questionnaire is for anyone not starting a university course this year 
(September / October 2006). 



11 Do you have a guaranteed / deferred place at university starting in Sept / October 2007? 



Yes 40 go to Q12 (n = 285) No 60 


go to Q14 (n 


= 427) 


Listed below are various things people do in ‘gap years’. For each of these, 
you think you will do this activity during the next 12 months? 

(Please S one box in each row.) 


please tell us if 




Yes 


No 


Don’t know 


Re-sit AS / A2 modules 


10 


82 


8 


Do a foundation / access course at FE college 


0 


88 


6 


Do charity / volunteer / unpaid work in UK 


26 


52 


22 


Do charity / volunteer / unpaid work abroad 


36 


44 


20 


Do paid work in UK 


93 


5 


3 


Do paid work abroad 


30 


47 


24 


Travel (for pleasure - not work / project related) 


72 


15 


13 



13 What is your main reason for taking a gap year? (Please S one box.) 
I want time to decide what I want to do next. 

So that I can earn some money to help finance my time at university. 

I want a break from studying. 

I want to travel. 

I want to do voluntary work (oversees or UK). 

I want to gain work experience (voluntary or paid). 

Now go to Q22 on page 10. 



13ji 



17J 2 



I29ja 






3J 5 



13je 



5 




This section of the questionnaire is for anyone not considering higher education in 2006 or 2007. 



14 Please indicate what you are doing at the moment - your main activity? (Please </ one box.) 



Apprenticeship (or other government funded training) 



0 , ? 



In a full-time job (30 hours or more per week) 



25 



2 



In a part-time job - less than 30 hrs per week 
(if this is your main activity) 

Self-employed 





go to Q16 
(n = 298) 



In full-time education at school / college 



21 



5 



Enrolled on a part-time education or training course, 
(excluding leisure courses) as your main activity 




J 



Looking for work / unemployed 




Taking a break from study or work 
Looking after a home or family 



0 , 

0 . 




6 



15 There may be specific reasons why you are not currently in employment, education or training. 
Please indicate whether or not the following reasons apply to you. 

(Please / one box in each row ) 





Yes 


No 


1 need more qualifications to get a job or place in education / training. 


16 


84 


1 am looking after the home / children or other family members. 


9 


91 


1 have a disability or poor health. 


m 


99 


There are no decent jobs where 1 live. 


23 


77 


1 have not yet decided what job or course 1 want to do. 


50 


50 


1 have not yet found a suitable job or course. 


76 


24 


1 have housing problems. 


□ 


100 


1 would be worse off financially in work or on a course. 


3 


97 


Travelling to work or college would be difficult because of poor 
transport where 1 live. 


3 


97 



16 How likely is it that you will apply for a place at university at some time in the future? 



Definitely will 
Probably will 
Probably won’t 
Definitely won’t 



52 



go to Q18 (n = 281) 



20 



22 



go to Q17 (n = 110) 



7 




17 Listed below are a number of reasons that some people give for not going to university. 
Please tell us how important each of these reasons was in your decision? 

(Please / one box in each row ) 



My A level grades are not good enough. 


Very 

important 

11 


Somewhat 

important 

27 


Not 

important 

62 


1 do not want to study any more. 


33 


37 


30 


1 cannot afford to go. 


30 


39 


31 


1 do not have the confidence to go to university. 


8 


23 


69 


1 do not need a degree for the job 1 wish to do. 


38 


27 


35 


1 applied but was unable to obtain a place. 


2 


3 


95 


1 think the work would be too difficult. 


6 


19 


75 


1 am worried about getting into debt. 


38 


35 


27 


1 do not wish to move away from my family or friends. 


9 


35 


56 


It’s difficult to know what it will be like. 


0 


30 


63 


1 don’t know how to apply. 


2 


0 


91 


I’m happy in the job that 1 am doing. 


30 


28 


43 



18 Did your results on the SAT® test that you took in year 13 as part of this research project: 
(Please / one box.) 

Encourage you to think about applying to university? 0 
Discourage you from applying to university? 0 



Make no difference to your plans for the future? 



92 



8 




Are you currently on an apprenticeship scheme, in full-time paid employment (30 hours or 
more per week) or waiting to take up a job / apprenticeship? 



Yes 



37 



go to Q20 (n = 146) 



No 



37 



go to Q22 (n = 246) 



In which of the following occupations or areas of work are you currently employed, on an 
apprenticeship scheme or waiting to take up work? (Please S one box.) 



Management (or trainee management) 

(eg industrial / office / retail / farm / hospitality / leisure management) 




Technical support posts 

(eg laboratory / electronics / science / engineering / IT technician) 




Health and social welfare 

(eg nurse, paramedic, dispensing optician, therapist) 




Protective service occupations 

(eg armed services, police, fire service, prison service) 




Culture, media and sports 

(eg actor, musician, journalist, photographer, fitness instructor) 




Business / public service occupations 

(eg insurance broker, buyer, sales representative, estate agent, careers adviser) 




Administrative and secretarial 

(eg, secretary, accounts clerk, (hotel) receptionsist, VDU operator, civil service EO) 



Skilled trades 

(eg electricians, mechanics, engineers, plumbers, printers) 




Construction trades 

(eg carpenters, bricklayers, glaziers, plasterers, decorators) 



Service occupations 

(eg dental nurse, care assistant, hairdresser, sales assistant, call centre operator) 




Machine operators / transport 

(eg bakery assistant, assembler, tyre fitter, machinist, drive) 



m 





21 What is your current gross annual salary before tax, Nl or any other deductions? 

(This question will allow us to monitor the starting salaries of non-graduates and compare 
these with the starting salaries of graduates in 2009.) 

(Please / one box.) 



Less than £10,000 a year 
Between £10,000 and £12,999 a year 
Between £13,000 and £16,999 a year 
Between £17,000 and £20,999 a year 
Between £21,000 and £25,000 a year 
More than £25,000 a year 






30 



2 



17 



3 



0 

I 

3 



This section is for everyone. 

22 We would like to contact you again during the next three years to find out about your 
experience of life after school / college. On each occasion we would like to offer a small 
token of our appreciation for your participation in this research. 



Please indicate which of these options would most encourage you to complete one of our 
surveys: 

(Please / one box.) 



Free gift (e.g. mouse-mat, mug, pen, etc) sent with questionnaire 
Music / book token - value £5 sent with questionnaire 
Prize draw (5 prizes of £1000) 

Prize draw (50 prizes of £100) 









I25ja 



[25j 4 



If you would like your name to be entered into the 2006 prize draw, 
please tick this box. 



97 



If you are one of the five winners and you would be willing to be 
interviewed for a news feature on our website, please tick this box. 



56 



10 





Thank you very much for your help 

Please put your questionnaire in the envelope provided and post it back to us 



as soon as possible. 



11 



© 2006 National Foundation for Educational Research 



Appendix 3: SAT® DescriptiveStatistics 

Theanalysesare basedon up to 9022studentswhotookthe 
SAT® test in autumn 2005. 




Table A3. 1: SAT Reasoning Test™ scores by gender 





N 


Critical Heading 


Mathematics 


Writing 


Male 


4167 


485.7 


516.3 


485.2 


Female 


4854 


493.1 


478.5 


506.5 


Total 


9021 









Values significantly different at (at least) the 5 per cent level are shown bold and in italics. 



Figure A3. 1 : Critical reading score distributions by gender 
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Figure A3.2: Mathematics score distributions by gender 




in re A3.3: Writing score distributions by gender 
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Table A3 .2: SAT® Critical Reading - total score distribution by gender 



Score 

Range 


Overall 

% 


Male 

% 


Female 

% 


200-224 


2.02 


2.21 


1.85 


225-249 


0.33 


0.41 


0.27 


250-274 


1.87 


2.23 


1.57 


275-299 


1.26 


1.32 


1.22 


300-324 


3.33 


3.48 


3.17 


325-349 


3.34 


3.96 


2.80 


350-374 


5.84 


6.38 


5.38 


375-399 


3.07 


3.24 


2.93 


400-424 


9.07 


9.05 


9.09 


425-449 


4.88 


4.30 


5.38 


450-474 


8.97 


9.24 


8.74 


475-499 


5.33 


4.92 


5.69 


500-524 


10.74 


9.82 


11.54 


525-549 


7.34 


7.06 


7.58 


550-574 


7.40 


7.20 


7.58 


575-599 


4.70 


4.68 


4.72 


600-624 


7.93 


8.11 


7.77 


625-649 


3.01 


2.69 


3.30 


650-674 


4.66 


4.58 


4.72 


675-699 


1.87 


2.04 


1.73 


700-724 


1.53 


1.37 


1.67 


725-749 


0.63 


0.72 


0.56 


750-774 


0.64 


0.77 


0.54 


775-800 


0.24 


0.24 


0.25 



n = 9022 

mean score = 489.63 
standard deviation =118.05 
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Table A3 .3: SAT® Mathematics - total score distribution by gender 



Score 

Range 


Overall 

% 


Male 

% 


Female 

% 


200-224 


1.06 


0.86 


1.24 


225-249 


0.42 


0.31 


0.52 


250-274 


1.31 


1.06 


1.50 


275-299 


2.25 


1.99 


2.47 


300-324 


3.01 


2.57 


3.40 


325-349 


2.98 


2.54 


3.36 


350-374 


4.02 


3.05 


4.86 


375-399 


4.88 


4.32 


5.36 


400-424 


7.79 


6.50 


8.90 


425-449 


6.06 


5.26 


6.76 


450-474 


11.46 


9.91 


12.79 


475-499 


5.89 


5.21 


6.47 


500-524 


9.64 


9.14 


10.07 


525-549 


4.80 


4.94 


4.68 


550-574 


8.41 


9.55 


7.44 


575-599 


4.17 


4.87 


3.56 


600-624 


6.52 


7.54 


5.64 


625-649 


2.95 


3.58 


2.41 


650-674 


5.32 


6.60 


4.22 


675-699 


2.52 


3.29 


1.85 


700-724 


2.67 


3.89 


1.63 


725-749 


0.41 


0.60 


0.25 


750-774 


0.75 


1.25 


0.33 


775-800 


0.70 


1.18 


0.29 



n = 9022 

mean score =495.93 
standard deviation =118.68 
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Table A3 .4: SAT® Writing - total score distribution by gender 



Score 

Range 


Overall 

% 


Male 

% 


Female 

% 


200-224 


0.67 


1.01 


0.37 


225-249 


0.35 


0.53 


0.19 


250-274 


0.65 


0.62 


0.68 


275-299 


0.78 


1.06 


0.54 


300-324 


1.36 


1.70 


1.07 


325-349 


1.61 


2.11 


1.17 


350-374 


3.45 


4.06 


2.93 


375-399 


4.37 


4.80 


4.00 


400-424 


7.57 


8.40 


6.86 


425-449 


6.47 


6.70 


6.28 


450-474 


12.66 


12.98 


12.38 


475-499 


7.94 


8.38 


7.56 


500-524 


13.47 


13.89 


13.10 


525-549 


9.30 


8.98 


9.58 


550-574 


9.21 


8.69 


9.66 


575-599 


6.99 


5.50 


8.28 


600-624 


5.60 


4.61 


6.45 


625-649 


2.84 


2.54 


3.09 


650-674 


2.80 


2.18 


3.34 


675-699 


0.85 


0.46 


1.19 


700-724 


0.61 


0.55 


0.66 


725-749 


0.24 


0.12 


0.35 


750-774 


0.11 


0.10 


0.12 


775-800 


0.10 


0.05 


0.14 



n = 9022 

mean score =496.61 
standard deviation =92.27 
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Table A3. 5: SAT Reasoning Test™ scores by type of institution 





N 


Critical 

Reading 


Mathematics 


Writing 


Comprehensive 


4784 


462.8 


462.2 


474.4 


Grammar 


1734 


540.6 


550.1 


536.8 


Independent 


2082 


514.8 


533.0 


519.8 


FE College 


422 


460.0 


473.2 


468.2 



Columns which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at (at least) the 5 per cent level. 



Figure A3.4: Critical reading score distributions by type of institution 




-Independent 
— - FE College 
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Figure A3.5: Mathematics score distributions by type of institution 




*ure A3.6: Writing score distributions by type of institution 




200-249 250-299 300-349 350-399 400-449 450-499 500-549 550-599 600-649 650-699 700-749 750-800 



Score range 
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Table A3. 6: SAT® Critical Reading - total score distribution by type 
of institution 



Score 

Range 


Overall 

% 


Comprehensive Grammar Independent 

% % % 


FE College 

% 


200-224 


2.02 


2.30 


0.69 


1.63 


6.16 


225-249 


0.33 


0.40 


0.17 


0.34 


0.24 


250-274 


1.87 


2.45 


0.23 


1.68 


3.08 


275-299 


1.26 


1.42 


0.23 


1.59 


2.13 


300-324 


3.33 


4.10 


1.10 


3.22 


4.27 


325-349 


3.34 


4.16 


1.15 


3.07 


4.27 


350-374 


5.84 


7.27 


3.34 


4.42 


6.87 


375-399 


3.07 


4.14 


1.67 


1.97 


2.13 


400-424 


9.07 


11.60 


4.73 


6.92 


8.77 


425-449 


4.88 


5.87 


3.63 


3.55 


5.21 


450-474 


8.97 


10.16 


7.79 


7.25 


8.77 


475-499 


5.33 


5.75 


5.77 


4.08 


4.98 


500-524 


10.74 


10.91 


11.19 


10.47 


8.29 


525-549 


7.34 


6.71 


9.52 


7.11 


6.64 


550-574 


7.40 


6.48 


10.15 


7.40 


6.64 


575-599 


4.70 


3.68 


6.63 


5.09 


6.40 


600-624 


7.93 


5.92 


11.65 


9.89 


5.69 


625-649 


3.01 


2.05 


4.04 


4.37 


3.08 


650-674 


4.66 


2.59 


7.09 


7.35 


4.74 


675-699 


1.87 


0.92 


3.58 


2.88 


0.71 


700-724 


1.53 


0.63 


2.77 


2.83 


0.24 


725-749 


0.63 


0.25 


1.21 


1.15 


0.00 


750-774 


0.64 


0.23 


1.04 


1.25 


0.71 


775-800 


0.24 


0.02 


0.63 


0.48 


0.00 



n = 9022 

mean score = 489.63 
standard deviation = 118.05 
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Table A3 .7 : SAT® Mathematics - total score distribution by type of 
institution 



Score 

Range 


Overall 

% 


Comprehensive Grammar 

% % 


Independent 

% 


IK 

College 

% 


200-224 


1.06 


1.36 


0.12 


0.72 


3.32 


225-249 


0.42 


0.63 


0.00 


0.19 


0.95 


250-274 


1.31 


1.96 


0.29 


0.67 


1.18 


275-299 


2.25 


3.24 


0.63 


1.39 


1.90 


300-324 


3.01 


4.29 


0.63 


1.73 


4.74 


325-349 


2.98 


4.06 


1.10 


1.83 


4.27 


350-374 


4.02 


5.39 


1.73 


2.74 


4.27 


375-399 


4.88 


6.77 


2.19 


2.64 


5.45 


400-424 


7.79 


10.05 


4.04 


5.86 


7.11 


425-449 


6.06 


7.36 


3.86 


4.90 


6.16 


450-474 


11.46 


12.37 


10.50 


10.09 


11.85 


475-499 


5.89 


6.00 


6.29 


5.62 


4.27 


500-524 


9.64 


9.05 


10.90 


9.94 


9.72 


525-549 


4.80 


4.12 


6.52 


4.90 


4.98 


550-574 


8.41 


6.73 


10.50 


10.52 


8.53 


575-599 


4.17 


3.22 


5.71 


4.76 


5.69 


600-624 


6.52 


5.18 


8.54 


7.97 


6.16 


625-649 


2.95 


1.96 


5.13 


3.65 


1.66 


650-674 


5.32 


3.22 


9.28 


7.20 


3.55 


675-699 


2.52 


1.42 


4.33 


3.79 


1.18 


700-724 


2.67 


0.98 


4.96 


4.80 


1.90 


725-749 


0.41 


0.23 


0.46 


0.82 


0.24 


750-774 


0.75 


0.21 


1.50 


1.44 


0.47 


775-800 


0.70 


0.19 


0.81 


1.83 


0.47 



n = 9022 

mean score = 495.93 
standard deviation = 118.68 
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Table A3 .8: SAT® Writing - total score distribution by type of 

institution 



Score 

Range 


Overall 

% 


Comprehensive Grammar Independent 

% % % 


FE College 

% 


200-224 


0.67 


0.65 


0.06 


0.82 


2.61 


225-249 


0.35 


0.38 


0.06 


0.48 


0.71 


250-274 


0.65 


0.56 


0.23 


0.91 


2.13 


275-299 


0.78 


0.96 


0.23 


0.67 


1.42 


300-324 


1.36 


1.86 


0.06 


1.30 


1.42 


325-349 


1.61 


2.09 


0.52 


1.20 


2.61 


350-374 


3.45 


4.24 


1.21 


3.03 


5.69 


375-399 


4.37 


6.02 


1.10 


2.83 


6.64 


400-424 


7.57 


10.41 


3.06 


5.00 


6.64 


425-449 


6.47 


8.26 


3.86 


3.99 


9.24 


450-474 


12.66 


15.20 


9.23 


9.89 


11.61 


475-499 


7.94 


9.01 


7.27 


6.05 


7.82 


500-524 


13.47 


13.09 


16.55 


11.96 


12.56 


525-549 


9.30 


8.24 


12.11 


9.75 


7.58 


550-574 


9.21 


7.19 


13.55 


10.81 


6.40 


575-599 


6.99 


4.70 


10.78 


9.32 


5.92 


600-624 


5.60 


3.47 


8.48 


8.50 


3.55 


625-649 


2.84 


1.71 


4.27 


4.47 


1.66 


650-674 


2.80 


1.17 


4.21 


5.57 


1.90 


675-699 


0.85 


0.36 


1.38 


1.49 


1.18 


700-724 


0.61 


0.27 


1.27 


0.86 


0.47 


725-749 


0.24 


0.08 


0.23 


0.62 


0.24 


750-774 


0.11 


0.02 


0.17 


0.29 


0.00 


775-800 


0.10 


0.06 


0.12 


0.19 


0.00 



n = 9022 

mean score = 496.61 
standard deviation = 92.27 
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T able A3.9 : SAT Reasoning T est™ scores by ethnicity 



Ethnicity 


N 


Critical 

Reading 


Mathematics 


Writing 


Chinese 


195 


409.5 


594.6 


408.2 


Black or Black British 


157 


431.4 


449.6 


449.0 


Asian or Asian British 


721 


436.7 


483.4 


453.6 


Other 


115 


466.6 


504.7 


472.8 


Mixed 


155 


490.0 


490.3 


497.8 


White 


6494 


501.9 


495.5 


507.0 



Columns which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at (at least) the 5 per cent level. 



Table A3. 10: SAT Reasoning Test™ scores by level of parental 

education 



Highest level of 
education 


N 


Critical 

Reading 


Mathematics 


Writing 


Mother- don’t know 


386 


454.0 


475.6 


465.2 


Mother -to 16 


2139 


479.8 


485.8 


488.8 


Mother -to 18 


1672 


498.6 


502.6 


505.6 


Mother - HE 


2397 


534.2 


535.2 


529.9 


Father- don’t know 


411 


453.9 


470.7 


463.8 


Father- to 16 


2055 


479.8 


480.9 


489.5 


Father- to 18 


1206 


493.1 


502.1 


501.1 


Father- HE 


2800 


532.1 


535.1 


528.4 



Columns which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at (at least) the 5 per cent level. 
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Table A3.ll: SAT Reasoning Test™ scores by parental occupation 



Parental occupation 

(NS-SEC analytic classes) 


N 


Critical 

Reading 


Mathematics 


Writing 


1.1 Large employers and 










higher managerial 
occupations 


767 


521.8 


522.0 


521.0 


1.2 Higher professional 
occupations 


1332 


543.8 


547.8 


538.2 


2 Lower managerial and 










professional 

occupations 


1718 


510.6 


511.4 


515.4 


3 Intermediate 
occupations 


576 


498.5 


505.3 


504.7 


4 Small employers and 
own account workers 


616 


470.5 


479.8 


482.4 


5 Lower supervisory and 
technical occupations 


342 


474.5 


477.3 


483.0 


6 Semi -routine 
occupations 


376 


467.2 


472.7 


477.2 


7 Routine occupations 

8 Unemployed (includes 


223 


462.1 


469.8 


472.1 


housewives, students, 
retired, etc) 


282 


468.1 


495.9 


474.2 



Columns which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at (at least) the 5 per cent level. 



Table A3. 12: SAT Reasoning Test™ scores by intention to go to 



university 





N 


Critical 

Reading 


Mathematics 


Writing 


Definitely not 


181 


428.1 


444.0 


448.2 


Probably not 


335 


431.4 


433.1 


452.2 


Probably 


1657 


475.5 


472.1 


481.6 


Definitely 


4597 


518.9 


526.6 


520.2 


Total 


6770 









Columns which are shown bold and in italics imply that for this outcome some categories are significantly 
different from others at (at least) the 5 per cent level. 
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Appendix4: Functioning of the SAT® 

The analysesare basedon upto 9022 students who tookthe 
SAT® test in autumn 2005. 




Functioning of the SAT® 



A number of analyses were earned out to examine the way in which the test was 
functioning, as a complete test and at the individual item level. Where data was 
available comparisons of the US and UK samples were made to see if there were any 
differences in performance that might indicate that the SAT® test was unsuitable for 
the English sample. All of the analyses reported here are based on 9022 UK students. 
The US sample comprises 288,905 students that took the same version of the SAT® 
test in March 2005. 

Englishsample 

Facilities and omission rates 

There were only seven items across the test as a whole where the facility (the 
percentage of test takers getting an item correct) was below 20 per cent and 11 items 
with omission rates above 20 per cent, indicating that the level of difficulty was 
appropriate for the sample. Almost all the items with omission rates above 20 per cent 
were amongst those items requiring student-produced responses in the first 
mathematics section (i.e. those with no multiple-choice options). A number of these 
items were classified as high difficulty items and therefore would be expected to 
produce low facilities and high omission rates. However, a small number of medium 
difficulty items in this section were also omitted by over 30 per cent of students. As 
these occurred at the end of the first mathematics section it may be that students had 
not paced themselves sufficiently or that students were put off by the unfamiliar grid 
format for recording their answers. 

Discrimination 

The discrimination of each item was also examined. The discrimination index for an 
item shows how well that item distinguishes between good and poorer performance, 
as measured by the test as a whole. It is calculated as the correlation between success 
on the item and the total score achieved on the remainder of the test and has a value 
between -1 and +1. A low discri mi nation may indicate that the item is measuring a 
different construct from the others in the test; items with very low discriminations 
detract from the internal consistency of the test. Almost all the items in the Critical 
Reading and Mathematics sections showed good discrimination. Writing items 
generally displayed acceptable levels of discrimination, although several items had 
discrimination values below 0.30 (generally considered to be a desirable value). 

Reliability 

For a test to be reliable it must be consistent in measuring the underlying construct 
(e.g. mathematical ability). The consistency of each section of the test was calculated 
using Cronbach’s Coefficient Alpha - shown as a figure up to one. The closer the 
coefficient is to one, the more confidert one can be that all the items are measuring 
the same construct. The results for the Critical Reading, Mathematics and Writing 
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sections of the SAT® were 0.94, 0.93 and 0.87 respectively. These are in line with the 
US figures and are acceptable reliabilites taking into account the number of items. 

Bias 



Against the background of the mean overall scores, analyses of differential item 
functioning klif) were performed, comparing male and female students and ‘Whites’ 
with all ‘Non- whites’ on each section of the SAT®. Dif analyses are used to check for 
potential bias by identifying any differences in performance on particular items 
between two groups of pupils, taking their total scores into account. In other words 
they may indicate items which are disproportiorately easy or difficult for a particular 
group of students. Screening of SAT® items for bias is earned out routinely in the US 
before items are included in live versions and during the earlier pilot very little 
evidence of bias was discovered amongst an English sample (McDonald et al., 
2001a). However, due to revisions to the SAT® since that time it was felt prudent to 
carry out dif analyses with the current sample. 

In the three dif analyses comparing male and female students, only two items in the 
Critical Reading section and one in the Writing section showed a large degree of bias, 
all three items in favour of male students. In the comparison of ‘Whites’ and ‘Non 
whites’, three items were found to exhibit a large degree of bias. In the Mathematics 
section one item favoured ‘Whites’ w hi lst another favoured the ‘Non-whites’ group 
and in the Writing section one item favoured ‘Non- whites’. A simple reading of the 
questions exhibiting bias did not suggest any obvious reasons for differential 
performance on these items, except for the Mathematics item favouring ‘Whites’. In 
this question it is possible that the words ‘pail’ and ‘gallons’ may have been less 
familiar to students from ethnic minority backgrounds. 

Correlations between components 

Correlation coefficients between scores on the various sections of the SAT®, 
including the two writing sub-scores, are shown in Table A4.1. All of these scores 
were positively correlated at a statistically significant level. Correlation coefficients 
indicate the association between two variables and therefore the extent to which the 
values of one variable can be predicted from the values of another. For example 
scores on the Critical Reading and Writing sections of the SAT® were highly 
correlated (0.80) indicating that these sections may be measuring similar constructs or 
abilities and that scores on one section arc highly predictive of scores on the other. 
The correlation between the Mathematics section and the other two sections were 
more modest (0.57 and 0.50 respectively). The lowest correlations were between the 
essay sub-score and the other sections of the test, although as you would expect the 
essay correlated more highly with reading and writing than with mathematics. 
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Table A4.1: Correlations between SAT® scores 





Critical 

Reading 

score 


Mathematics 

score 


Writing 

score 


Writing m/c 
sub-score 


Essay 
sub- score 


Critical 

Reading 


1.00 


0.57 


0.80 


0.81 


0.37 


score 












Mathematics 

score 


0.57 


1.00 


0.50 


0.54 


0.14 


Writing 

score 


0.80 


0.50 


1.00 


0.97 


0.59 


Writing m/c 
sub-score 


0.81 


0.54 


0.97 


1.00 


0.37 


Essay sub- 
score 


0.37 


0.14 


0.59 


0.37 


1.00 
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Comparison of English and USsamples 

Table A4.2 shows the means obtained by the English and US student groups on each 
of the main components of the SAT®. 

Table A4.2: Comparison of mean scores of American and English 
students 



SAT® section 


US 

Mean 


s.d. 


England 

Mean 


s.d. 


Critical reading 


526 


110 


490 


118 


Mathematics 


544 


111 


496 


119 


Writing 


525 


110 


497 


92 


Number of students 


288905 




9022 





Although there are clear differences in the means of the two groups of students, a 
direct comparison of English and US means cannot be made. Due to the high stakes 
associated with the test in the US (almost every college considers SAT® scores as 
pail of its admission process), it is likely that American students were generally more 
highly motivated than English students. It can also be assumed, for the same reason 
that US students would have prepared more thoroughly than their English 
counterparts and therefore may have been more fa mil iar with the types of questions 
and the foimat of the SAT® answer sheets (particularly the grids for entering student 
produced responses within one of the mathematics sections). According to the College 
Board these particular US students turned out to be a relatively high scorhg group. In 
the US, students can choose when to take the SAT® and, as March 2005 was the first 
administration of the new style SAT® containing the essay component, this appeared 
to attract a more able group than is usually the case. The means for over 1.4 million 
US high school graduates over the full year to April 2006 (and therefore using a 
number of different versions of the SAT®) were much closer to the UK means at 503, 
518 and 497 for reading, mathematics and writing respectively (College Board, 2006) 

Item functioning 

In the 2001 pilot the SAT® test proved to perform similarly with a sample of English 
students compared to students in the US. However, as the SAT Reasoning TesF M has 
undergone some changes since that time, a comparison of the results ofAmerican and 
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English students was carried out at the individual item level to see if there were any 
differences in performance that might indicate that the SAT® test was unsuitable for 
the English sample. This consisted of an analysis of item facilities (he percentage of 
test takers getting an item correct) and o mi ssion rates (the percentage of students 
omitting an item) for the two groups. 

Comparison of Item Facilities for US and English Candidates 

ETS provided NFER with a spreadsheet showing the item characteristics for each of 
the eight sections of multiple choice items. 



The table below describes briefly the characteristics of the eight sections analysed. 

Table A4.3: Number of items per SAT® section 



Section 


Description 


No. of items 


2 


Mathematics 2 


18 


3 


Writing 1 


35 


4 


Reading 2 


23 


5 


Mathematics 1 


20 


6 


Reading 1 


25 


7 


Mathematics 3 


16 


8 


Reading 3 


19 


9 


Writing 2 


14 



For the English data, facilities were computed for each item- the percentage of ah 
those doing the test who get the item right. For the US sample, a spreadsheet was 
provided with a variable PPLUS, which was described as being similar to the facility 
value. However, a detailed frequency breakdown for each item in the March 2005 
administration was also provided by ETS, from which it was possible to extract both 
facilities and percentage omitting the item, and therefore item facilities and omission 
rates for the two samples were compared using these values. 

The table below summaries the mean facilities for each section, and the correlations 
between English and US facility values. 
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Table A4.4: Comparison of SAT® facilities 



Section 


Description 


Mean US 
facility 


Mean English 
facility 


Correlation 


2 


Maths 2 


56 


50 


0.95 


3 


Writing 1 


60 


55 


0.88 


4 


Reading 2 


61 


55 


0.96 


5 


Maths 1 


64 


54 


0.95 


6 


Reading 1 


57 


53 


0.98 


7 


Maths 3 


59 


53 


0.96 


8 


Reading 3 


58 


51 


0.89 


9 


Writing 2 


70 


58 


0.96 



From the above it seemed that there was a fair measure of agreement at the item level 
between English and US performance, but that on most items the US facility was a 
few points higher than the English value. It should be noted that the derivation of 
scores from items is relatively complex - credit is subtracted for wrong answers, and 
this may have differentially affected the total scores for the two samples. To exarrine 
this in more detail, average omission rates for the two samples were compared. 

An interesting feature of the data was that omission rates for items in the US sample 
seemed to vary more within sections than was the case for the English sample. This is 
illustrated in the second of the two figures below, which shows omission rate standard 
deviations within sections. 



Appendix 4- page 6 







Figure A4.1: SAT® section omission rates 



SAT Section Omission Rates 

Writing 2 
Reading 3 
Maths 3 
Reading 1 
Maths 1 
Reading 2 
Writing 1 
Maths 2 

0% 5% 10% 15% 




□ English mean 

□ US Mean 



20% 



Figure A4.2: SAT® section omission rate standard deviations 
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As referred to above, the way in vhich SAT® scores are derived is complicated by 
the fact that on each multiple-choice item, a quarter- mark is deducted for each 
incorrect item (to discourage guessing). Although this would have been included in 
the instructions given to both groups, it is possible that English students may have 
overlooked this instruction, due to less preparation or motivation. The US variation in 
omission rates (particularly in the multiple- choice mathematics sections) suggests that 
American students were being more strategic in the items they omitted and this may 
have contributed to the difference in mean scores of the two groups. 

As individual student-level data for items in the US group was not available, it was 
not possible to cany out an analysis of differential item functioning (dif) in the normal 
way. A ‘pseudo-dif analysis was therefore earned out by converting facilities within 
each of the three main areas to the logrodds metric and normalising differences in log- 
odds to pick up items where the difference was bc\ond the approximate 95 per cent 
confidence interval. This analysis identified 11 items where the performance of the 
two groups was significantly different; six in the Reading section, two in Writing and 
three in Mathematics. Eight of these items appeared 0 favour students in the English 
sample and the remaining five the US sample. No immediately obvious reasons for 
such differences were apparent, with the exception of one item in the Reading section 
- where the selection of the word ‘archipelago’ best fit the meaning of a sentence 
about Hawaii. As might be expected performance on this item was higher amongst the 
US sample. 

Overall these results indicate that the individual SAT® items functioned in a similar 
way for the English and US samples. Detailed item statistics for the English sample 
can be found in tables A4.6 to A4.8 at the back of this appendix. 

Effect of preparation 

Each student participating in the SAT® trial was sent a test preparation booklet 
containing information about the types of questions in the test and how to fill in the 
answer grids for the student- produced response questions in the mathematics section. 
On the agreement form that students completed in order to participate in the study, a 
small number of questions were posed about the amount and type of practice that they 
had earned out in advance of taking the SAT®. 

Three-quarters of the students who completed the SAT® indicated that they had 
received a copy of the preparation booklet from their school or college and three per 
cent reported that they had looked at the preparation materials on the College Board 
website. The amount of time students spent in preparing for the SAT® test is shown 
in Table A4.5. 



Appendix 4- page 8 




Table A4.5:Time spent by students in preparation for the SAT® test 





Number 


Per cent 


Mean SAT® scores 








Reading 


Mathematics 


Writing 


No preparation 


3327 


36.9 


484.0 


492.6 


494.6 


Less than 15 
minutes 


1776 


19.7 


512.2 


516.3 


512.4 


15-29 minutes 


1070 


11.9 


507.6 


510.2 


506.9 


30-59 minutes 


625 


6.9 


504.8 


506.7 


505.2 


1-2 hours 


220 


2.4 


501.6 


511.9 


504.6 


More than 2 hours 


58 


0.6 


443.5 


468.8 


467.4 


Missing 


1946 


21.6 








Total 


9022 


100.0 









Columns which are shown bold and in italics imply that for this outcome some 
categories are significantly different from others at (at least) the 5 per cent level. 

Over a third of the students who took the SAT® earned out no preparation 
whatsoever and less than a quarter spent more than 15 minutes preparing for the test. 
When asked to indicate how they spent their preparation time just over a third of all 
students spent some time reading through the sample questions in the Critical Reading 
section, which was the first section in the preparation booklet. Less than twenty per 
cent of students claimed to have looked at the sample essays in the final section of he 
preparation booklet. 

The mean scores of students who claimed to have carried out no preparation were 
lower than each of the preparation groups (with the exception of the group of students 
who claimed to have spent more than 2 hours preparing for the test), although not all 
of the differences were statistically significant and the relationship between the 
amount of preparation and mean scores was not linear. The highest mean in each 
section was achieved by the group who had spent less than 15 minutes in preparing 
for the SAT®. The very small group of students, who indicated a preparation time of 
over two hours, achieved significantly lower mean scores than all the other groups. It 
is possible that this group of students had not taken the test or the questions about 
preparation seriously and had given a facetious response. Alternatively, it is possible 
that these were diligent but much less able students. Overall the amount of preparation 
does not appear to have had a significant effect on the scores achic\cd. although a 
small amount of fa mi liarisation was beneficial compared to no preparation 
whatsoever. 
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Table A4.6:Item analysis of Critical Reading section 



Item 


Facility 
(% correct) 


Percentage 

Omitted 


Discrimation 


SATIT401 


80 


2 


0.31 


SATIT402 


29 


5 


0.28 


SATIT403 


59 


2 


0.43 


SATIT404 


28 


5 


0.23 


SATIT405 


21 


4 


0.33 


SATIT406 


84 


2 


0.33 


SATIT407 


73 


2 


0.41 


SATIT408 


30 


3 


0.25 


SATIT409 


14 


5 


0.18 


SATIT410 


62 


2 


0.38 


SATIT41 1 


56 


4 


0.36 


SATIT412 


77 


3 


0.50 


SATIT413 


45 


4 


0.34 


SATIT414 


56 


4 


0.33 


SATIT415 


68 


8 


0.49 


SATIT416 


63 


7 


0.46 


SATIT417 


56 


10 


0.38 


SATIT418 


77 


8 


0.50 


SATIT419 


52 


12 


0.47 


SATIT420 


65 


12 


0.50 


SATIT421 


54 


14 


0.54 


SATIT422 


61 


17 


0.42 


SATIT423 


41 


19 


0.38 
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Table A4.6:Item analysis of Critical Reading section (continued) 



Item 


Facility 
(% correct) 


Percentage Omitted 


Discrimation 


SATIT601 


82 


4 


0.40 


SATIT602 


55 


5 


0.53 


SATIT603 


42 


5 


0.53 


SATIT604 


59 


5 


0.31 


SATIT605 


41 


8 


0.38 


SATIT606 


31 


10 


0.36 


SATIT607 


40 


6 


0.52 


SATIT608 


24 


11 


0.19 


SATIT609 


69 


5 


0.46 


SATIT610 


75 


4 


0.48 


SATIT61 1 


66 


5 


0.54 


SATIT612 


39 


7 


0.48 


SATIT613 


33 


6 


0.39 


SATIT614 


48 


6 


0.38 


SATIT615 


57 


6 


0.47 


SATIT616 


73 


6 


0.54 


SATIT617 


75 


6 


0.53 


SATIT618 


75 


6 


0.54 


SATIT619 


48 


8 


0.41 


SATIT620 


24 


9 


0.35 


SATIT621 


50 


10 


0.44 


SATIT622 


60 


9 


0.50 


SATIT623 


68 


12 


0.58 


SATIT624 


54 


12 


0.50 


SATIT625 


47 


15 


0.51 
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Table A4.6:Item analysis of Critical Reading section (continued) 



Item 


Facility 
(% correct) 


Percentage 

Omitted 


Discrimation 


SATIT801 


77 


6 


0.51 


SATIT802 


74 


6 


0.41 


SATIT803 


30 


10 


0.28 


SATIT804 


51 


7 


0.45 


SATIT805 


33 


6 


0.43 


SATIT806 


29 


11 


0.41 


SATIT807 


57 


7 


0.49 


SATIT808 


68 


7 


0.44 


SATIT809 


68 


7 


0.51 


SATIT810 


28 


8 


0.22 


SATIT811 


64 


8 


0.52 


SATIT812 


42 


9 


0.44 


SATIT813 


54 


9 


0.49 


SATIT814 


21 


10 


0.25 


SATIT815 


51 


10 


0.47 


SATIT816 


49 


12 


0.43 


SATIT817 


53 


12 


0.47 


SATIT818 


62 


13 


0.54 


SATIT819 


53 


15 


0.49 
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Table A4.7: Item analysis of Mathematics section 



Item 


Facility 
(% correct) 


Percentage Omitted 


Discrimation 


SATIT201 


90 


1 


0.35 


SATIT202 


87 


1 


0.27 


SATIT203 


87 


1 


0.30 


SATIT204 


69 


2 


0.44 


SATIT205 


41 


7 


0.39 


SATIT206 


38 


8 


0.44 


SATIT207 


43 


6 


0.38 


SATIT208 


23 


11 


0.46 


SATIT209 


85 


2 


0.28 


SATIT210 


67 


6 


0.42 


SATIT211 


61 


22 


0.47 


SATIT212 


58 


9 


0.46 


SATIT213 


31 


33 


0.60 


SATIT214 


22 


21 


0.43 


SATIT215 


35 


35 


0.54 


SATIT216 


29 


26 


0.53 


SATIT217 


28 


34 


0.56 


SATIT218 


9 


48 


0.39 


SATIT501 


88 


3 


0.37 


SATIT502 


79 


4 


0.50 


SATIT503 


79 


3 


0.37 


SATIT504 


69 


3 


0.43 


SATIT505 


80 


5 


0.44 


SATIT506 


74 


3 


0.45 


SATIT507 


64 


7 


0.55 


SATIT508 


73 


3 


0.38 


SATIT509 


53 


9 


0.48 


SATIT510 


61 


7 


0.55 


SATIT511 


54 


9 


0.39 
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Table A4.7: Item analysis of Mathematics section (continued) 



Item 


Facility 
(% correct) 


Percentage Omitted 


Discrimation 


SATIT512 


40 


10 


0.46 


SATIT513 


43 


8 


0.47 


SATIT514 


38 


7 


0.57 


SATIT515 


33 


9 


0.45 


SATIT516 


52 


14 


0.49 


SATIT517 


38 


13 


0.42 


SATIT518 


31 


17 


0.35 


SATIT519 


27 


22 


0.48 


SATIT520 


13 


17 


0.18 


SATIT701 


81 


5 


0.36 


SATIT702 


83 


5 


0.45 


SATIT703 


76 


6 


0.41 


SATIT704 


61 


9 


0.47 


SATIT705 


80 


6 


0.47 


SATIT706 


56 


7 


0.59 


SATIT707 


61 


7 


0.44 


SATIT708 


46 


10 


0.54 


SATIT709 


55 


8 


0.49 


SATIT710 


46 


10 


0.46 


SATIT711 


52 


7 


0.35 


SATIT712 


32 


16 


0.47 


SATIT713 


38 


17 


0.49 


SATIT714 


33 


15 


0.37 


SATIT715 


19 


21 


0.24 


SATIT716 


22 


26 


0.35 
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Table A4.8: Item analysis of Writing section 



Item 


Facility 
(% correct) 


Percentage Omitted 


Discrimation 


SATIT301 


75 


1 


0.29 


SATIT302 


80 


1 


0.26 


SATIT303 


82 


1 


0.24 


SATIT304 


53 


1 


0.38 


SATIT305 


78 


1 


0.22 


SATIT306 


44 


2 


0.33 


SATIT307 


58 


1 


0.30 


SATIT308 


46 


1 


0.20 


SATIT309 


45 


1 


0.25 


SATIT310 


22 


2 


0.16 


SATIT3 1 1 


53 


1 


0.33 


SATIT312 


92 


1 


0.29 


SATIT313 


21 


2 


0.22 


SATIT314 


90 


1 


0.26 


SATIT315 


78 


1 


0.32 


SATIT316 


61 


2 


0.31 


SATIT317 


76 


1 


0.36 


SATIT318 


65 


2 


0.41 


SATIT319 


86 


2 


0.33 


SATIT320 


13 


2 


0.19 


SATIT321 


69 


2 


0.27 


SATIT322 


55 


3 


0.24 


SATIT323 


26 


4 


0.24 


SATIT324 


78 


3 


0.38 


SATIT325 


23 


4 


0.30 


SATIT326 


18 


5 


0.28 


SATIT327 


21 


6 


0.25 


SATIT328 


48 


6 


0.43 


SATIT329 


66 


7 


0.34 
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Table A4.8: Item analysis of Writing section (continued) 



Item 


Facility 
(% correct) 


Percentage 

Omitted 


Discrimation 


SATIT330 


51 


8 


0.42 


SATIT331 


53 


10 


0.37 


SATIT332 


54 


12 


0.28 


SATIT333 


53 


15 


0.41 


SATIT334 


53 


17 


0.43 


SATIT335 


47 


21 


0.34 


SATIT901 


87 


5 


0.40 


SATIT902 


79 


5 


0.43 


SATIT903 


84 


6 


0.46 


SATIT904 


77 


6 


0.41 


SATIT905 


74 


6 


0.48 


SATIT906 


70 


6 


0.45 


SATIT907 


73 


6 


0.47 


SATIT908 


58 


8 


0.41 


SATIT909 


45 


7 


0.40 


SATIT910 


42 


7 


0.30 


SATIT911 


35 


7 


0.37 


SATIT912 


45 


12 


0.32 


SATIT913 


27 


13 


0.23 


SATIT914 


18 


15 


0.17 
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